ODC and Dask (LocalCluster)

• Setup
• Load data without dask
• Exploring dask concepts with ODC
• The impact of dask on ODC
• Exploiting delayed tasks
• Data and computational locality
  • With compute on the algorithm
  • With selected measurements
• A quick check on the task graph

This notebook explores the use of ODC with Dask LocalCluster. The goal is to introduce fundamental concepts and the role Dask can serve with datacube and subsequent computation using xarray.

The example computation is fairly typical of an EO data processing pipeline. We'll be using a small area and time period to start with and progressively scaling this example. EO scientists may find some aspects of these examples unrealistic, but this isn't an EO science course ☺.

The basic workflow is:

1. Specify Region of Interest, Satellite product, EO satellite bands, Time range, desired CRS for the datacube query
2. Load data using datacube.load()
3. Mask valid data
4. Visualisation of the ROI
5. Compute NDVI
6. Visualise NDVI

NOTE: Some cells in this notebook will take minutes to run so please be patient. Also, some cells can load large datasets into Jupyter's memory (based on the defaults), which can exhaust the available memory and cause the kernel to crash. If this occurs: restart the kernal, run the Setup cells, and then jump to the next section. The (text) outputs from previous sections can be retained between kernel restarts. Example times for each exercise:

Process	N times	Load
Load data without dask	7	3-3.5 mins
Exploring dask concepts with ODC	2	1 min
The impact of dask on ODC	22	1-2 mins
Exploiting delayed tasks	7	1 min
Data and computational locality	7	30-50 secs
With compute on the algorithm	7	10 secs
With selected measurements	7	10 secs

Setup

import git
import sys, os
from dateutil.parser import parse
from dateutil.relativedelta import relativedelta
from dask.distributed import Client, LocalCluster
import datacube
from datacube.utils import masking
from datacube.utils.aws import configure_s3_access

# EASI defaults
os.environ['USE_PYGEOS'] = '0'
repo = git.Repo('.', search_parent_directories=True).working_tree_dir
if repo not in sys.path: sys.path.append(repo)
from easi_tools import EasiDefaults, notebook_utils
easi = EasiDefaults()
client = None

Successfully found configuration for deployment "csiro"

The next cell sets out all the query parameters used in our datacube.load(). For this run we keep the ROI quite small.

# Get the default latitude & longitude extents
study_area_lat = easi.latitude
study_area_lon = easi.longitude

# Or choose your own by uncommenting and modifying this section
###############################################################
# # Central Tasmania (near Little Pine Lagoon)
# central_lat = -42.019
# central_lon = 146.615

# # Set the buffer to load around the central coordinates
# # This is a radial distance for the bbox to actual area so bbox 2x buffer in both dimensions
# buffer = 0.05

# # Compute the bounding box for the study area
# study_area_lat = (central_lat - buffer, central_lat + buffer)
# study_area_lon = (central_lon - buffer, central_lon + buffer)
###############################################################

# Data product
product = easi.product('landsat')
# product = 'landsat8_c2l2_sr'

# Set the date range to load data over
set_time = easi.time
# set_time = ("2021-01-01", "2021-01-31")

# Selected measurement names (used in this notebook). None` will load all of them
alias = easi.aliases('landsat')
measurements = None
# measurements = [alias[x] for x in ['qa_band', 'red', 'nir']]

# Set the QA band name and mask values
qa_band = alias['qa_band']
qa_mask = easi.qa_mask('landsat')

# Set the resampling method for the bands
resampling = {qa_band: "nearest", "*": "average"}

# Set the coordinate reference system and output resolution
set_crs = easi.crs('landsat')                # If defined, else None
set_resolution = easi.resolution('landsat')  # If defined, else None
# set_crs = "epsg:3577"
# set_resolution = (-30, 30)

# Set the scene group_by method
group_by = "solar_day"

Now initialise the datacube.

dc = datacube.Datacube()

# Access AWS "requester-pays" buckets
# This is necessary for reading data from most third-party AWS S3 buckets such as for Landsat and Sentinel-2
from datacube.utils.aws import configure_s3_access
configure_s3_access(aws_unsigned=False, requester_pays=True);

Load data without dask

Now load the data. This first dc.load() does not use Dask so it will take a few minutes.

We use %%time to keep track of how long things take to complete.

%%time
dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            group_by=group_by,
        )

CPU times: user 1min 25s, sys: 12.5 s, total: 1min 38s
Wall time: 3min 36s

The result of the datacube.load() function is an xarray.Dataset.

Jupyter notebooks can render a description of the xarray dataset variable with a lot of useful information about the structure of data.

dataset

<xarray.Dataset>
Dimensions:                     (time: 7, y: 2013, x: 1709)
Coordinates:
  * time                        (time) datetime64[ns] 2020-02-08T23:56:55.714...
  * y                           (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06
  * x                           (x) float64 1.328e+06 1.328e+06 ... 1.379e+06
    spatial_ref                 int32 3577
Data variables: (12/22)
    nbart_coastal_aerosol       (time, y, x) int16 -999 -999 -999 ... 972 973
    nbart_blue                  (time, y, x) int16 -999 -999 -999 ... 978 980
    nbart_green                 (time, y, x) int16 -999 -999 -999 ... 1053 1082
    nbart_red                   (time, y, x) int16 -999 -999 -999 ... 1119 1155
    nbart_nir                   (time, y, x) int16 -999 -999 -999 ... 2676 2631
    nbart_swir_1                (time, y, x) int16 -999 -999 -999 ... 1912 2136
    ...                          ...
    oa_relative_slope           (time, y, x) float32 26.82 25.31 ... 95.97 114.5
    oa_satellite_azimuth        (time, y, x) float32 100.1 100.1 ... 100.2 100.2
    oa_satellite_view           (time, y, x) float32 9.022 9.02 ... 3.36 3.358
    oa_solar_azimuth            (time, y, x) float32 67.9 67.9 ... 47.08 47.08
    oa_solar_zenith             (time, y, x) float32 39.49 39.49 ... 50.96 50.96
    oa_time_delta               (time, y, x) float32 -0.4956 -0.4971 ... 5.429
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

• Dimensions:
  • time: 7
  • y: 2013
  • x: 1709
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 ... 2...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000',
           '2020-02-24T23:56:51.968040000', '2020-03-03T00:03:00.297499000',
           '2020-03-11T23:56:45.657477000', '2020-03-19T00:02:52.920718000',
           '2020-03-27T23:56:36.905801000'], dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Data variables: (22)
  • nbart_coastal_aerosol
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 941 972 973

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1824, 1832, 1861],
            [-999, -999, -999, ..., 1812, 1826, 1858],
            [-999, -999, -999, ..., 1801, 1819, 1846],
            ...,
            [4977, 4976, 4969, ..., 3373, 3424, 3403],
            [4946, 4947, 4948, ..., 3421, 3497, 3461],
            [4918, 4947, 4946, ..., 3507, 3553, 3487]],
    
           [[ 868,  860,  862, ...,  644,  627,  630],
            [ 827,  857,  857, ...,  615,  618,  648],
            [ 815,  831,  838, ...,  608,  624,  651],
            ...,
            [ 468,  350,  327, ...,  642,  644,  623],
            [ 394,  328,  310, ...,  569,  561,  545],
            [ 353,  327,  311, ...,  440,  478,  499]],
    
           [[-999, -999, -999, ...,  736,  731,  733],
            [-999, -999, -999, ...,  708,  709,  740],
            [-999, -999, -999, ...,  698,  717,  746],
            ...,
    ...
            ...,
            [ 309,  309,  310, ...,  490,  498,  475],
            [ 308,  300,  303, ...,  423,  440,  433],
            [ 325,  337,  336, ...,  328,  345,  390]],
    
           [[1231, 1310, 1339, ...,  521,  528,  569],
            [1171, 1279, 1348, ...,  486,  493,  535],
            [1151, 1216, 1272, ...,  510,  513,  526],
            ...,
            [2027, 1876, 1673, ...,  711,  707,  706],
            [2061, 1955, 1731, ...,  664,  646,  630],
            [2139, 2016, 1800, ...,  624,  617,  603]],
    
           [[-999, -999, -999, ...,  480,  461,  462],
            [-999, -999, -999, ...,  443,  441,  451],
            [-999, -999, -999, ...,  427,  427,  436],
            ...,
            [ 340,  344,  347, ..., 1015, 1026,  992],
            [ 333,  340,  344, ...,  988, 1014,  988],
            [ 356,  384,  383, ...,  941,  972,  973]]], dtype=int16)

  • nbart_blue
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 949 978 980

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1854, 1868, 1889],
            [-999, -999, -999, ..., 1839, 1858, 1885],
            [-999, -999, -999, ..., 1833, 1851, 1881],
            ...,
            [4951, 4963, 4941, ..., 3358, 3411, 3387],
            [4919, 4932, 4933, ..., 3408, 3481, 3444],
            [4901, 4937, 4937, ..., 3488, 3543, 3470]],
    
           [[1074, 1070, 1070, ...,  764,  740,  739],
            [ 999, 1027, 1019, ...,  729,  732,  759],
            [ 978,  995, 1005, ...,  718,  739,  769],
            ...,
            [ 467,  322,  296, ...,  776,  753,  716],
            [ 376,  289,  278, ...,  641,  640,  615],
            [ 324,  287,  283, ...,  482,  491,  534]],
    
           [[-999, -999, -999, ...,  868,  856,  856],
            [-999, -999, -999, ...,  831,  840,  872],
            [-999, -999, -999, ...,  823,  846,  867],
            ...,
    ...
            ...,
            [ 331,  330,  329, ...,  572,  562,  545],
            [ 332,  325,  321, ...,  476,  493,  506],
            [ 344,  373,  371, ...,  347,  388,  420]],
    
           [[1275, 1356, 1402, ...,  530,  524,  555],
            [1216, 1307, 1387, ...,  501,  501,  529],
            [1197, 1247, 1301, ...,  529,  533,  536],
            ...,
            [2021, 1912, 1697, ...,  788,  780,  763],
            [2063, 1991, 1764, ...,  707,  704,  694],
            [2149, 2046, 1835, ...,  641,  619,  628]],
    
           [[-999, -999, -999, ...,  531,  502,  489],
            [-999, -999, -999, ...,  491,  483,  486],
            [-999, -999, -999, ...,  473,  473,  474],
            ...,
            [ 356,  364,  369, ..., 1053, 1044, 1018],
            [ 350,  361,  351, ...,  996, 1035, 1020],
            [ 386,  416,  402, ...,  949,  978,  980]]], dtype=int16)

  • nbart_green
    (time, y, x)
    int16
    -999 -999 -999 ... 1047 1053 1082

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1971, 1982, 2003],
            [-999, -999, -999, ..., 1950, 1963, 1993],
            [-999, -999, -999, ..., 1942, 1958, 1985],
            ...,
            [4932, 4943, 4925, ..., 3364, 3423, 3400],
            [4903, 4922, 4912, ..., 3417, 3488, 3455],
            [4890, 4923, 4912, ..., 3505, 3558, 3483]],
    
           [[1635, 1584, 1574, ..., 1074, 1040, 1030],
            [1458, 1479, 1466, ..., 1011, 1015, 1047],
            [1416, 1429, 1445, ..., 1000, 1023, 1047],
            ...,
            [ 486,  335,  295, ..., 1047, 1003,  940],
            [ 405,  292,  275, ...,  833,  811,  790],
            [ 349,  294,  285, ...,  569,  635,  682]],
    
           [[-999, -999, -999, ..., 1187, 1151, 1148],
            [-999, -999, -999, ..., 1111, 1122, 1156],
            [-999, -999, -999, ..., 1104, 1134, 1152],
            ...,
    ...
            ...,
            [ 419,  413,  415, ...,  833,  803,  757],
            [ 420,  403,  404, ...,  649,  681,  696],
            [ 462,  483,  487, ...,  447,  485,  558]],
    
           [[1621, 1659, 1635, ...,  781,  782,  823],
            [1555, 1642, 1682, ...,  687,  697,  758],
            [1525, 1604, 1663, ...,  675,  707,  743],
            ...,
            [2063, 1829, 1697, ..., 1040,  985,  920],
            [2095, 1880, 1723, ...,  894,  875,  834],
            [2143, 1967, 1768, ...,  697,  760,  802]],
    
           [[-999, -999, -999, ...,  788,  758,  745],
            [-999, -999, -999, ...,  735,  718,  741],
            [-999, -999, -999, ...,  707,  715,  735],
            ...,
            [ 445,  447,  449, ..., 1233, 1222, 1176],
            [ 443,  437,  428, ..., 1155, 1171, 1137],
            [ 511,  529,  501, ..., 1047, 1053, 1082]]], dtype=int16)

  • nbart_red
    (time, y, x)
    int16
    -999 -999 -999 ... 1030 1119 1155

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2137, 2148, 2168],
            [-999, -999, -999, ..., 2111, 2122, 2158],
            [-999, -999, -999, ..., 2103, 2123, 2150],
            ...,
            [4912, 4938, 4919, ..., 3382, 3447, 3421],
            [4883, 4911, 4901, ..., 3446, 3527, 3483],
            [4880, 4915, 4905, ..., 3538, 3596, 3514]],
    
           [[2268, 2184, 2160, ..., 1366, 1316, 1279],
            [1989, 2010, 2008, ..., 1276, 1268, 1305],
            [1927, 1938, 1961, ..., 1257, 1275, 1313],
            ...,
            [ 486,  328,  294, ..., 1339, 1273, 1198],
            [ 386,  274,  252, ..., 1042, 1016,  959],
            [ 332,  279,  273, ...,  665,  738,  845]],
    
           [[-999, -999, -999, ..., 1561, 1509, 1483],
            [-999, -999, -999, ..., 1464, 1466, 1483],
            [-999, -999, -999, ..., 1444, 1453, 1459],
            ...,
    ...
            ...,
            [ 427,  436,  446, ...,  919,  889,  860],
            [ 432,  423,  426, ...,  774,  761,  767],
            [ 469,  536,  535, ...,  464,  558,  730]],
    
           [[1854, 1909, 1900, ...,  682,  668,  696],
            [1794, 1884, 1935, ...,  574,  591,  640],
            [1765, 1829, 1896, ...,  582,  599,  627],
            ...,
            [2118, 1897, 1744, ..., 1068, 1015,  978],
            [2146, 1967, 1779, ...,  929,  949,  911],
            [2205, 2054, 1833, ...,  777,  818,  893]],
    
           [[-999, -999, -999, ...,  778,  700,  651],
            [-999, -999, -999, ...,  699,  657,  641],
            [-999, -999, -999, ...,  668,  637,  621],
            ...,
            [ 449,  468,  476, ..., 1237, 1230, 1218],
            [ 439,  457,  445, ..., 1173, 1181, 1204],
            [ 525,  573,  561, ..., 1030, 1119, 1155]]], dtype=int16)

  • nbart_nir
    (time, y, x)
    int16
    -999 -999 -999 ... 2617 2676 2631

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2823, 2837, 2861],
            [-999, -999, -999, ..., 2797, 2813, 2845],
            [-999, -999, -999, ..., 2788, 2806, 2829],
            ...,
            [5169, 5179, 5158, ..., 3927, 3985, 3957],
            [5143, 5160, 5150, ..., 3988, 4065, 4018],
            [5133, 5160, 5150, ..., 4078, 4127, 4049]],
    
           [[3545, 3377, 3295, ..., 2795, 2795, 2833],
            [2969, 2972, 2930, ..., 2687, 2724, 2807],
            [2848, 2847, 2869, ..., 2664, 2713, 2723],
            ...,
            [1514, 1132,  987, ..., 2529, 2356, 2150],
            [1309,  915,  867, ..., 2271, 2186, 2169],
            [1173,  820,  789, ..., 2082, 2238, 2334]],
    
           [[-999, -999, -999, ..., 2778, 2764, 2792],
            [-999, -999, -999, ..., 2688, 2729, 2803],
            [-999, -999, -999, ..., 2694, 2755, 2786],
            ...,
    ...
            ...,
            [1918, 1830, 1795, ..., 2346, 2355, 2243],
            [1966, 1825, 1782, ..., 2215, 2209, 2164],
            [1967, 1842, 1779, ..., 1846, 2002, 2082]],
    
           [[2933, 3015, 3026, ..., 2553, 2619, 2788],
            [2812, 2928, 3009, ..., 2447, 2482, 2709],
            [2767, 2846, 2929, ..., 2452, 2505, 2664],
            ...,
            [3604, 3421, 3238, ..., 2905, 2785, 2575],
            [3628, 3491, 3281, ..., 2704, 2601, 2490],
            [3692, 3569, 3337, ..., 2351, 2419, 2414]],
    
           [[-999, -999, -999, ..., 3328, 3438, 3559],
            [-999, -999, -999, ..., 3262, 3386, 3632],
            [-999, -999, -999, ..., 3200, 3390, 3625],
            ...,
            [2199, 2117, 2057, ..., 2992, 3053, 2854],
            [2217, 2108, 2067, ..., 2910, 2989, 2818],
            [2209, 2091, 2009, ..., 2617, 2676, 2631]]], dtype=int16)

  • nbart_swir_1
    (time, y, x)
    int16
    -999 -999 -999 ... 1821 1912 2136

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1958, 1966, 1988],
            [-999, -999, -999, ..., 1947, 1956, 1983],
            [-999, -999, -999, ..., 1947, 1952, 1978],
            ...,
            [1747, 1758, 1744, ..., 1696, 1724, 1709],
            [1730, 1733, 1716, ..., 1726, 1766, 1743],
            [1702, 1707, 1687, ..., 1780, 1803, 1761]],
    
           [[4288, 4196, 4151, ..., 3679, 3685, 3739],
            [3848, 3908, 3925, ..., 3643, 3688, 3780],
            [3768, 3813, 3860, ..., 3667, 3685, 3769],
            ...,
            [ 971,  695,  588, ..., 3403, 3494, 3418],
            [ 769,  525,  457, ..., 2754, 3002, 2980],
            [ 638,  478,  447, ..., 1830, 1883, 2178]],
    
           [[-999, -999, -999, ..., 3812, 3801, 3867],
            [-999, -999, -999, ..., 3739, 3785, 3867],
            [-999, -999, -999, ..., 3749, 3766, 3820],
            ...,
    ...
            ...,
            [1274, 1350, 1322, ..., 2850, 2928, 2927],
            [1254, 1314, 1226, ..., 2319, 2634, 2724],
            [1326, 1465, 1424, ..., 1499, 1664, 1896]],
    
           [[3479, 3486, 3448, ..., 1910, 1855, 1978],
            [3402, 3484, 3502, ..., 1695, 1699, 1846],
            [3374, 3461, 3522, ..., 1623, 1631, 1714],
            ...,
            [2896, 2706, 2618, ..., 2836, 2871, 2828],
            [2927, 2710, 2628, ..., 2399, 2568, 2616],
            [2960, 2770, 2656, ..., 1831, 1978, 2150]],
    
           [[-999, -999, -999, ..., 2253, 2239, 2210],
            [-999, -999, -999, ..., 2210, 2200, 2173],
            [-999, -999, -999, ..., 2224, 2177, 2150],
            ...,
            [1322, 1407, 1401, ..., 2621, 2701, 2732],
            [1257, 1380, 1290, ..., 2336, 2507, 2523],
            [1385, 1544, 1482, ..., 1821, 1912, 2136]]], dtype=int16)

  • nbart_swir_2
    (time, y, x)
    int16
    -999 -999 -999 ... 1193 1322 1393

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1590, 1599, 1617],
            [-999, -999, -999, ..., 1583, 1592, 1618],
            [-999, -999, -999, ..., 1585, 1583, 1610],
            ...,
            [1933, 1938, 1925, ..., 1643, 1677, 1664],
            [1904, 1912, 1901, ..., 1677, 1721, 1696],
            [1878, 1894, 1881, ..., 1735, 1763, 1718]],
    
           [[3109, 3091, 3087, ..., 2571, 2546, 2554],
            [3010, 3036, 3074, ..., 2537, 2549, 2634],
            [2940, 2991, 3045, ..., 2555, 2563, 2686],
            ...,
            [ 541,  351,  305, ..., 2207, 2358, 2308],
            [ 397,  254,  226, ..., 1788, 1917, 1954],
            [ 319,  247,  244, ...,  990, 1184, 1331]],
    
           [[-999, -999, -999, ..., 2690, 2657, 2677],
            [-999, -999, -999, ..., 2620, 2638, 2707],
            [-999, -999, -999, ..., 2623, 2619, 2677],
            ...,
    ...
            ...,
            [ 638,  697,  705, ..., 1770, 1859, 1899],
            [ 610,  671,  629, ..., 1509, 1584, 1653],
            [ 721,  857,  831, ...,  777, 1022, 1293]],
    
           [[2955, 2998, 2959, ..., 1190, 1148, 1260],
            [2892, 3002, 3040, ...,  998,  992, 1089],
            [2846, 2955, 3043, ...,  934,  949,  992],
            ...,
            [2495, 2295, 2206, ..., 1817, 1843, 1810],
            [2535, 2305, 2228, ..., 1559, 1620, 1652],
            [2566, 2396, 2252, ..., 1106, 1297, 1430]],
    
           [[-999, -999, -999, ..., 1344, 1297, 1246],
            [-999, -999, -999, ..., 1284, 1254, 1219],
            [-999, -999, -999, ..., 1287, 1231, 1199],
            ...,
            [ 641,  706,  725, ..., 1653, 1726, 1757],
            [ 602,  703,  684, ..., 1476, 1583, 1691],
            [ 770,  894,  874, ..., 1193, 1322, 1393]]], dtype=int16)

  • nbart_panchromatic
    (time, y, x)
    int16
    -999 -999 -999 ... 992 1029 1033

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2007, 2012, 2038],
            [-999, -999, -999, ..., 1968, 1988, 2034],
            [-999, -999, -999, ..., 1970, 1982, 2029],
            ...,
            [4886, 4887, 4879, ..., 3382, 3442, 3396],
            [4841, 4846, 4866, ..., 3439, 3529, 3472],
            [4823, 4857, 4853, ..., 3524, 3604, 3520]],
    
           [[1965, 1849, 1795, ..., 1170, 1085, 1095],
            [1595, 1604, 1611, ..., 1049, 1066, 1106],
            [1574, 1555, 1602, ..., 1066, 1068, 1120],
            ...,
            [ 553,  298,  273, ..., 1185, 1093,  995],
            [ 362,  253,  228, ...,  925,  919,  881],
            [ 321,  266,  273, ...,  583,  598,  621]],
    
           [[-999, -999, -999, ..., 1306, 1227, 1240],
            [-999, -999, -999, ..., 1189, 1204, 1239],
            [-999, -999, -999, ..., 1191, 1196, 1234],
            ...,
    ...
            ...,
            [ 394,  397,  400, ...,  836,  802,  741],
            [ 387,  380,  357, ...,  686,  725,  739],
            [ 405,  492,  513, ...,  423,  458,  473]],
    
           [[1583, 1664, 1750, ...,  645,  591,  632],
            [1508, 1616, 1713, ...,  584,  613,  632],
            [1502, 1553, 1585, ...,  653,  628,  650],
            ...,
            [2126, 2053, 1803, ..., 1010,  992,  945],
            [2170, 2143, 1912, ...,  886,  909,  917],
            [2256, 2203, 1980, ...,  746,  707,  709]],
    
           [[-999, -999, -999, ...,  732,  654,  647],
            [-999, -999, -999, ...,  629,  636,  635],
            [-999, -999, -999, ...,  635,  610,  623],
            ...,
            [ 411,  426,  424, ..., 1212, 1211, 1143],
            [ 407,  404,  385, ..., 1122, 1181, 1189],
            [ 441,  509,  491, ...,  992, 1029, 1033]]], dtype=int16)

  • oa_fmask
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 2 2 2 2 2 2 2 2

    units :

    1

    nodata :

    0

    flags_definition :

    {'fmask': {'bits': [0, 1, 2, 3, 4, 5, 6, 7], 'values': {'0': 'nodata', '1': 'valid', '2': 'cloud', '3': 'shadow', '4': 'snow', '5': 'water'}, 'description': 'Fmask'}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            ...,
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 3, 3, ..., 0, 0, 0],
            [2, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
    ...
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 2, 2, ..., 1, 1, 1],
            [2, 2, 2, ..., 3, 2, 1],
            [2, 2, 2, ..., 2, 1, 1],
            ...,
            [2, 2, 2, ..., 0, 0, 0],
            [2, 2, 2, ..., 0, 0, 0],
            [2, 2, 2, ..., 0, 0, 0]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 2, 2, 2],
            [1, 1, 1, ..., 2, 2, 2],
            [1, 1, 1, ..., 2, 2, 2]]], dtype=uint8)

  • oa_nbart_contiguity
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    flags_definition :

    {'contiguous': {'bits': [0], 'values': {'0': False, '1': True}}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
    ...
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_azimuthal_exiting
    (time, y, x)
    float32
    -80.12 -69.12 ... 22.35 37.82

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -80.11553  ,  -69.122116 ,  -74.337006 , ...,   33.770046 ,
               97.201294 ,  133.90964  ],
            [ -98.09167  ,  -94.05396  ,  -87.45524  , ..., -135.61787  ,
              -76.76864  ,   76.9891   ],
            [-102.43699  , -115.73346  , -107.09284  , ..., -145.66154  ,
             -141.03618  , -102.94009  ],
            ...,
            [  18.39366  ,   13.645186 ,   17.93953  , ...,   60.979168 ,
               83.76587  ,   66.93567  ],
            [  19.606314 ,   11.669298 ,   11.353658 , ...,   88.5983   ,
               61.46648  ,   47.17998  ],
            [  11.797114 ,    2.8835816,    3.9532704, ...,   91.169556 ,
               22.845282 ,   38.14933  ]],
    
           [[  68.512245 ,   70.72293  ,   63.153164 , ...,  -57.384872 ,
              -57.144466 ,   -8.937615 ],
            [  57.491165 ,   53.669292 ,   51.882816 , ...,   62.990677 ,
               32.850204 , -101.98451  ],
            [  41.615105 ,   28.149704 ,   29.797287 , ...,   56.238945 ,
               50.34515  ,  -35.11987  ],
    ...
            [ -55.702927 ,  -61.211754 ,  -58.602432 , ...,   42.026577 ,
               31.557383 ,   14.543039 ],
            [ -56.799675 ,  -58.752808 ,  -55.27524  , ...,   35.831524 ,
              -21.98377  ,  -20.66293  ],
            [ -52.400978 ,  -51.16223  ,  -45.72211  , ...,  -17.596964 ,
              -73.51931  ,  -37.414417 ]],
    
           [[ -79.99074  ,  -68.98204  ,  -74.266914 , ...,   33.75241  ,
               97.28622  ,  134.04129  ],
            [ -98.037865 ,  -93.850296 ,  -87.26825  , ..., -135.74379  ,
              -76.8089   ,   76.99631  ],
            [-102.430214 , -115.63545  , -106.95093  , ..., -145.8245   ,
             -141.16183  , -102.985214 ],
            ...,
            [  18.327644 ,   13.5717945,   17.836235 , ...,   60.792133 ,
               83.2655   ,   66.5012   ],
            [  19.505722 ,   11.564668 ,   11.242666 , ...,   88.150795 ,
               60.84028  ,   46.80816  ],
            [  11.684288 ,    2.7711146,    3.8519013, ...,   89.46725  ,
               22.353941 ,   37.816006 ]]], dtype=float32)

  • oa_azimuthal_incident
    (time, y, x)
    float32
    -53.29 -43.81 ... 118.3 117.2

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -53.29337 ,  -43.814816,  -49.677975, ...,   73.61261 ,
               83.28998 , -150.34354 ],
            [ -70.001015,  -67.43494 ,  -62.690357, ...,  -89.084045,
              -35.280712,  110.770065],
            [ -77.07651 ,  -90.91911 ,  -84.01256 , ...,  -97.06677 ,
             -102.89715 ,  -62.93791 ],
            ...,
            [  71.86952 ,   65.37238 ,   70.617676, ...,  -37.93936 ,
             -127.73148 ,   55.0821  ],
            [  72.86513 ,   63.776108,   64.33563 , ...,  -86.956635,
               92.59873 ,  138.75403 ],
            [  65.48859 ,   55.617256,   58.376175, ..., -145.5778  ,
               96.301636,  128.84294 ]],
    
           [[ -49.41655 ,  -40.01738 ,  -45.558876, ...,   77.89437 ,
               87.38188 , -146.33037 ],
            [ -65.91871 ,  -64.03974 ,  -59.186226, ...,  -84.63618 ,
              -31.004824,  115.21529 ],
            [ -72.751   ,  -87.00204 ,  -80.26874 , ...,  -92.55007 ,
              -98.49117 ,  -58.568077],
    ...
            [  91.60852 ,   84.93896 ,   90.27645 , ...,  -18.902132,
             -107.43312 , -144.26265 ],
            [  92.57256 ,   83.466866,   84.186295, ...,  -67.536736,
              112.970085,   86.24169 ],
            [  85.45657 ,   75.66629 ,   78.73765 , ..., -125.76915 ,
              116.34627 ,  114.81904 ]],
    
           [[ -33.89823 ,  -24.55739 ,  -30.547585, ...,   93.728714,
              103.18652 , -130.67923 ],
            [ -50.51801 ,  -47.96289 ,  -43.424152, ...,  -68.75012 ,
              -15.084247,  130.70366 ],
            [ -57.913475,  -71.72347 ,  -64.95042 , ...,  -76.658714,
              -82.49213 ,  -42.690422],
            ...,
            [  94.1386  ,   87.42163 ,   92.70324 , ...,  -17.779425,
             -106.265755, -142.6202  ],
            [  95.02548 ,   85.95459 ,   86.72992 , ...,  -66.26677 ,
              114.41551 ,   88.39858 ],
            [  88.032005,   78.3464  ,   81.56147 , ..., -125.77241 ,
              118.32581 ,  117.21635 ]]], dtype=float32)

  • oa_combined_terrain_shadow
    (time, y, x)
    uint8
    1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
    ...
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_exiting_angle
    (time, y, x)
    float32
    9.029 9.148 9.09 ... 6.889 7.5

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 9.028952 ,  9.148497 ,  9.090177 , ...,  3.2647538,
              3.62325  ,  4.391917 ],
            [ 8.894047 ,  8.901706 ,  8.9225235, ...,  3.2806778,
              3.5062037,  4.4214954],
            [ 8.756343 ,  8.535571 ,  8.580602 , ...,  3.3433805,
              3.560637 ,  4.636894 ],
            ...,
            [11.375915 , 11.375726 , 11.181578 , ...,  5.0770392,
              4.6290197,  6.9061594],
            [11.1641655, 11.834425 , 12.261691 , ...,  3.2524686,
              5.314759 ,  7.5270324],
            [12.537902 , 13.934173 , 14.685269 , ...,  3.791945 ,
              6.8892117,  7.5075154]],
    
           [[ 2.4255953,  2.4921818,  2.615967 , ...,  7.997434 ,
              7.613728 ,  6.8861303],
            [ 2.4387937,  2.5729144,  2.7521896, ...,  8.068149 ,
              7.777022 ,  6.803534 ],
            [ 2.8313277,  3.079027 ,  3.2421515, ...,  8.0514555,
              7.821932 ,  6.652463 ],
    ...
            [ 5.4637203,  5.05376  ,  5.183171 , ..., 15.88036  ,
             13.218683 , 13.809436 ],
            [ 5.3021894,  5.4204   ,  5.9068046, ..., 12.913045 ,
             10.258273 , 11.662609 ],
            [ 6.269576 ,  7.0905676,  8.104537 , ...,  9.329348 ,
              8.103664 , 10.459551 ]],
    
           [[ 8.992556 ,  9.112733 ,  9.0536995, ...,  3.2260282,
              3.5847125,  4.353568 ],
            [ 8.857459 ,  8.866613 ,  8.8875475, ...,  3.242549 ,
              3.4677517,  4.3825727],
            [ 8.719783 ,  8.500572 ,  8.546016 , ...,  3.3054862,
              3.522601 ,  4.598103 ],
            ...,
            [11.340418 , 11.341775 , 11.150212 , ...,  5.0904117,
              4.6352854,  6.9089704],
            [11.13384  , 11.806992 , 12.236776 , ...,  3.25406  ,
              5.307453 ,  7.5122714],
            [12.512222 , 13.910551 , 14.6625595, ...,  3.8069766,
              6.888689 ,  7.4997563]]], dtype=float32)

  • oa_incident_angle
    (time, y, x)
    float32
    40.0 40.23 40.24 ... 47.95 45.56

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.998978, 40.23141 , 40.237175, ..., 37.26962 , 37.826603,
             38.7017  ],
            [39.77791 , 39.912277, 40.08276 , ..., 37.007404, 37.392212,
             38.432606],
            [39.88898 , 39.73803 , 39.905514, ..., 36.956375, 37.170578,
             38.289707],
            ...,
            [39.404842, 39.735092, 39.47829 , ..., 31.522882, 33.095287,
             32.230022],
            [39.361443, 39.853664, 39.843723, ..., 33.740055, 35.684074,
             34.42446 ],
            [39.81281 , 40.565845, 40.41699 , ..., 36.779434, 37.866028,
             35.635822]],
    
           [[40.41124 , 40.656933, 40.68124 , ..., 37.69816 , 38.263546,
             39.134884],
            [40.19523 , 40.337387, 40.529827, ..., 37.413853, 37.801704,
             38.827084],
            [40.35013 , 40.206455, 40.390034, ..., 37.354183, 37.558178,
             38.643627],
    ...
            [45.827156, 46.22167 , 45.993977, ..., 41.226646, 41.564674,
             39.83358 ],
            [45.863823, 46.168343, 45.974117, ..., 42.578743, 43.184433,
             41.239265],
            [45.820477, 46.122406, 45.623905, ..., 44.6841  , 44.740677,
             42.35347 ]],
    
           [[51.799583, 52.08417 , 52.133915, ..., 49.2278  , 49.80294 ,
             50.615532],
            [51.540302, 51.749332, 52.00923 , ..., 48.873055, 49.26349 ,
             50.1861  ],
            [51.81843 , 51.74149 , 51.986668, ..., 48.778515, 48.947083,
             49.863976],
            ...,
            [48.973274, 49.372353, 49.151676, ..., 44.756584, 44.991596,
             43.186226],
            [49.018948, 49.29584 , 49.077053, ..., 46.040924, 46.505142,
             44.49764 ],
            [48.909595, 49.148643, 48.6031  , ..., 48.018078, 47.950554,
             45.562294]]], dtype=float32)

  • oa_relative_azimuth
    (time, y, x)
    float32
    32.23 32.23 32.23 ... 53.09 53.09

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 32.232582,  32.23285 ,  32.233112, ...,  32.693703,
              32.69397 ,  32.69425 ],
            [ 32.23295 ,  32.23321 ,  32.233475, ...,  32.69407 ,
              32.69434 ,  32.694626],
            [ 32.233315,  32.233578,  32.23384 , ...,  32.694443,
              32.694706,  32.694992],
            ...,
            [ 32.975964,  32.976234,  32.976505, ...,  33.447357,
              33.447643,  33.447918],
            [ 32.976337,  32.976612,  32.976875, ...,  33.447735,
              33.448017,  33.44829 ],
            [ 32.976727,  32.976997,  32.977264, ...,  33.44813 ,
              33.448418,  33.44869 ]],
    
           [[216.49684 , 216.49713 , 216.49739 , ..., 216.99608 ,
             216.99635 , 216.99666 ],
            [216.4972  , 216.49747 , 216.49776 , ..., 216.99643 ,
             216.99673 , 216.99702 ],
            [216.49756 , 216.49783 , 216.49811 , ..., 216.9968  ,
             216.99709 , 216.99738 ],
    ...
            [230.64865 , 230.64899 , 230.6493  , ..., 231.19696 ,
             231.1973  , 231.19763 ],
            [230.64893 , 230.64925 , 230.64957 , ..., 231.19722 ,
             231.19754 , 231.19788 ],
            [230.6492  , 230.64952 , 230.64984 , ..., 231.19751 ,
             231.19785 , 231.19817 ]],
    
           [[ 52.08729 ,  52.087597,  52.087902, ...,  52.614693,
              52.614998,  52.61531 ],
            [ 52.087517,  52.087822,  52.088127, ...,  52.614925,
              52.615234,  52.615547],
            [ 52.087746,  52.088055,  52.088356, ...,  52.615154,
              52.615463,  52.61578 ],
            ...,
            [ 52.553524,  52.55383 ,  52.55414 , ...,  53.08538 ,
              53.085697,  53.086006],
            [ 52.553757,  52.55406 ,  52.55437 , ...,  53.085617,
              53.08593 ,  53.08624 ],
            [ 52.55401 ,  52.554314,  52.554626, ...,  53.085876,
              53.08619 ,  53.086502]]], dtype=float32)

  • oa_relative_slope
    (time, y, x)
    float32
    26.82 25.31 24.66 ... 95.97 114.5

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[  26.822166,   25.307304,   24.659027, ...,   39.842567,
               32.689396,   26.666718],
            [  28.09065 ,   26.619028,   24.764885, ...,   46.53382 ,
               41.487926,   33.780968],
            [  25.360477,   24.81435 ,   23.08028 , ...,   48.594772,
               47.248375,   40.002182],
            ...,
            [  53.47586 ,   51.727196,   52.678143, ..., -169.05045 ,
               43.374695,  110.16484 ],
            [  53.258812,   52.106808,   52.98198 , ...,  116.92698 ,
               94.31888 ,  100.24516 ],
            [  53.69147 ,   52.733677,   54.422905, ...,   82.8719  ,
               73.45635 ,   90.69361 ]],
    
           [[-117.928795, -110.74031 , -108.71204 , ..., -144.39311 ,
             -141.10318 , -137.39276 ],
            [-123.409874, -117.70903 , -111.069046, ..., -147.62685 ,
             -145.64392 , -142.8002  ],
            [-114.366104, -115.15174 , -110.066025, ..., -148.78902 ,
             -148.83632 , -148.09137 ],
    ...
            [ 147.31145 ,  146.1507  ,  148.87888 , ..., -131.06064 ,
             -145.53075 , -158.8057  ],
            [ 149.37224 ,  142.21967 ,  139.46153 , ..., -138.46336 ,
             -158.45459 , -170.31526 ],
            [ 137.85756 ,  126.828514,  124.459755, ..., -148.55292 ,
             -170.13441 , -172.67322 ]],
    
           [[  46.092506,   44.424652,   43.719326, ...,   59.976303,
               52.50102 ,   46.199368],
            [  47.51986 ,   45.88741 ,   43.844093, ...,   66.99367 ,
               61.724648,   53.707344],
            [  44.51674 ,   43.91198 ,   42.00051 , ...,   69.16578 ,
               67.77906 ,   60.29479 ],
            ...,
            [  75.81096 ,   73.84984 ,   74.867004, ..., -148.70349 ,
               65.3408  ,  134.07596 ],
            [  75.51976 ,   74.38992 ,   75.48725 , ...,  138.06435 ,
              116.76464 ,  124.37052 ],
            [  76.34772 ,   75.57528 ,   77.709564, ...,  104.379616,
               95.97187 ,  114.49366 ]]], dtype=float32)

  • oa_satellite_azimuth
    (time, y, x)
    float32
    100.1 100.1 100.1 ... 100.2 100.2

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[100.1332  , 100.13319 , 100.13318 , ..., 100.11447 ,
             100.114456, 100.11445 ],
            [100.13323 , 100.13322 , 100.13321 , ..., 100.1145  ,
             100.114494, 100.11449 ],
            [100.13326 , 100.13325 , 100.13324 , ..., 100.11453 ,
             100.114525, 100.11452 ],
            ...,
            [100.19855 , 100.19853 , 100.198524, ..., 100.179245,
             100.17924 , 100.17922 ],
            [100.19858 , 100.19857 , 100.198555, ..., 100.17928 ,
             100.179276, 100.17926 ],
            [100.198616, 100.1986  , 100.198586, ..., 100.17931 ,
             100.1793  , 100.17929 ]],
    
           [[280.09695 , 280.09692 , 280.09692 , ..., 280.07846 ,
             280.07843 , 280.07843 ],
            [280.097   , 280.09695 , 280.09695 , ..., 280.0785  ,
             280.0785  , 280.07846 ],
            [280.09702 , 280.09702 , 280.097   , ..., 280.07852 ,
             280.07852 , 280.0785  ],
    ...
            [280.1545  , 280.15448 , 280.15448 , ..., 280.13547 ,
             280.13547 , 280.13547 ],
            [280.15454 , 280.1545  , 280.1545  , ..., 280.1355  ,
             280.1355  , 280.1355  ],
            [280.15457 , 280.15454 , 280.15454 , ..., 280.13553 ,
             280.13553 , 280.13553 ]],
    
           [[100.12064 , 100.12063 , 100.12062 , ..., 100.10195 ,
             100.10194 , 100.10193 ],
            [100.120674, 100.12066 , 100.12065 , ..., 100.10199 ,
             100.101974, 100.10197 ],
            [100.120705, 100.1207  , 100.12068 , ..., 100.10201 ,
             100.102005, 100.102   ],
            ...,
            [100.18591 , 100.185905, 100.18589 , ..., 100.16665 ,
             100.16664 , 100.166626],
            [100.18594 , 100.185936, 100.18592 , ..., 100.16669 ,
             100.16668 , 100.166664],
            [100.18598 , 100.18597 , 100.18596 , ..., 100.16672 ,
             100.1667  , 100.166695]]], dtype=float32)

  • oa_satellite_view
    (time, y, x)
    float32
    9.022 9.02 9.017 ... 3.36 3.358

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[9.02193  , 9.019681 , 9.017433 , ..., 5.130571 , 5.1282773,
             5.125959 ],
            [9.021114 , 9.018867 , 9.016618 , ..., 5.1297407, 5.1274476,
             5.1251235],
            [9.0203   , 9.018051 , 9.015802 , ..., 5.128911 , 5.1266174,
             5.1242886],
            ...,
            [7.31853  , 7.3162613, 7.3139925, ..., 3.4020782, 3.3997495,
             3.3974464],
            [7.31763  , 7.3153787, 7.3131275, ..., 3.401222 , 3.3989155,
             3.3966124],
            [7.316651 , 7.314382 , 7.3121133, ..., 3.400168 , 3.397848 ,
             3.3955448]],
    
           [[2.1344194, 2.1367428, 2.1390662, ..., 6.0918517, 6.094148 ,
             6.096469 ],
            [2.1352227, 2.137546 , 2.1398695, ..., 6.092644 , 6.094941 ,
             6.097266 ],
            [2.1360261, 2.1383495, 2.140673 , ..., 6.0934367, 6.095733 ,
             6.0980635],
    ...
            [3.839198 , 3.841511 , 3.843824 , ..., 7.7688255, 7.7711244,
             7.7734   ],
            [3.8400757, 3.8423707, 3.8446655, ..., 7.7696314, 7.771909 ,
             7.7741847],
            [3.8410332, 3.8433464, 3.8456593, ..., 7.7706327, 7.7729244,
             7.7752   ]],
    
           [[8.984301 , 8.982052 , 8.979802 , ..., 5.0916533, 5.0893593,
             5.08704  ],
            [8.983486 , 8.981236 , 8.978987 , ..., 5.090824 , 5.08853  ,
             5.0862055],
            [8.982671 , 8.980421 , 8.978172 , ..., 5.0899944, 5.0877004,
             5.0853705],
            ...,
            [7.281676 , 7.279406 , 7.2771363, ..., 3.3641264, 3.3617976,
             3.3594937],
            [7.280776 , 7.278524 , 7.2762723, ..., 3.363271 , 3.3609638,
             3.3586605],
            [7.279797 , 7.277528 , 7.275258 , ..., 3.3622172, 3.3598967,
             3.3575928]]], dtype=float32)

  • oa_solar_azimuth
    (time, y, x)
    float32
    67.9 67.9 67.9 ... 47.08 47.08

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[67.90062 , 67.900345, 67.90006 , ..., 67.42077 , 67.42049 ,
             67.4202  ],
            [67.90028 , 67.90001 , 67.899734, ..., 67.42043 , 67.42015 ,
             67.41986 ],
            [67.89995 , 67.89967 , 67.8994  , ..., 67.4201  , 67.419815,
             67.41952 ],
            ...,
            [67.22258 , 67.2223  , 67.222015, ..., 66.73189 , 66.73159 ,
             66.73131 ],
            [67.222244, 67.221954, 67.22168 , ..., 66.73155 , 66.731255,
             66.73097 ],
            [67.221886, 67.2216  , 67.22132 , ..., 66.73118 , 66.73089 ,
             66.7306  ]],
    
           [[63.60011 , 63.59981 , 63.599514, ..., 63.082386, 63.082085,
             63.08177 ],
            [63.59979 , 63.59949 , 63.599194, ..., 63.082058, 63.081757,
             63.081444],
            [63.599464, 63.599167, 63.59887 , ..., 63.08173 , 63.08143 ,
             63.081112],
    ...
            [49.50584 , 49.505512, 49.505184, ..., 48.938507, 48.938168,
             48.93784 ],
            [49.505608, 49.505283, 49.504955, ..., 48.93828 , 48.937943,
             48.93761 ],
            [49.505363, 49.505035, 49.504707, ..., 48.93802 , 48.937683,
             48.93735 ]],
    
           [[48.03335 , 48.033035, 48.03272 , ..., 47.48726 , 47.48694 ,
             47.486618],
            [48.033157, 48.03284 , 48.032524, ..., 47.48706 , 47.486744,
             47.486416],
            [48.032963, 48.032642, 48.032326, ..., 47.486862, 47.48654 ,
             47.486214],
            ...,
            [47.632393, 47.632072, 47.631756, ..., 47.081272, 47.080944,
             47.08062 ],
            [47.632187, 47.63187 , 47.631554, ..., 47.081074, 47.080746,
             47.080425],
            [47.63197 , 47.631653, 47.631332, ..., 47.08084 , 47.080517,
             47.080193]]], dtype=float32)

  • oa_solar_zenith
    (time, y, x)
    float32
    39.49 39.49 39.49 ... 50.96 50.96

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.487972, 39.48771 , 39.48745 , ..., 39.04551 , 39.04525 ,
             39.044994],
            [39.48805 , 39.487785, 39.487526, ..., 39.045586, 39.045326,
             39.04507 ],
            [39.48812 , 39.48786 , 39.4876  , ..., 39.045662, 39.045403,
             39.045147],
            ...,
            [39.63633 , 39.63607 , 39.63581 , ..., 39.195976, 39.195724,
             39.19546 ],
            [39.636402, 39.636143, 39.635883, ..., 39.196053, 39.1958  ,
             39.19554 ],
            [39.636463, 39.636204, 39.635944, ..., 39.196114, 39.195854,
             39.195595]],
    
           [[39.84592 , 39.845665, 39.845406, ..., 39.414013, 39.413757,
             39.413506],
            [39.846012, 39.845757, 39.8455  , ..., 39.41411 , 39.413853,
             39.413605],
            [39.846107, 39.84585 , 39.845596, ..., 39.414204, 39.413948,
             39.4137  ],
    ...
            [47.97353 , 47.9733  , 47.973076, ..., 47.593254, 47.593037,
             47.59281 ],
            [47.973675, 47.97345 , 47.973225, ..., 47.593407, 47.59319 ,
             47.592964],
            [47.973816, 47.973587, 47.973362, ..., 47.593544, 47.593323,
             47.593098]],
    
           [[51.01565 , 51.01543 , 51.015205, ..., 50.641705, 50.641483,
             50.641266],
            [51.01581 , 51.015587, 51.015366, ..., 50.64186 , 50.64164 ,
             50.641426],
            [51.01597 , 51.015747, 51.015522, ..., 50.64202 , 50.6418  ,
             50.64159 ],
            ...,
            [51.331642, 51.33142 , 51.3312  , ..., 50.95994 , 50.95973 ,
             50.959507],
            [51.3318  , 51.331577, 51.331356, ..., 50.960102, 50.95989 ,
             50.959667],
            [51.331944, 51.331722, 51.3315  , ..., 50.960247, 50.960033,
             50.959812]]], dtype=float32)

  • oa_time_delta
    (time, y, x)
    float32
    -0.4956 -0.4971 ... 5.43 5.429

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-0.49559563, -0.4970854 , -0.49857512, ..., -2.9306152 ,
             -2.9321241 , -2.9334474 ],
            [-0.49143195, -0.49292168, -0.49441144, ..., -2.9264584 ,
             -2.9279675 , -2.929252  ],
            [-0.48726824, -0.48875797, -0.49024773, ..., -2.9223018 ,
             -2.923811  , -2.9250565 ],
            ...,
            [ 7.86643   ,  7.864933  ,  7.863436  , ...,  5.4181333 ,
              5.416802  ,  5.4152856 ],
            [ 7.870569  ,  7.869079  ,  7.867589  , ...,  5.422319  ,
              5.420964  ,  5.4194474 ],
            [ 7.874677  ,  7.8731804 ,  7.8716836 , ...,  5.4264264 ,
              5.4250336 ,  5.423517  ]],
    
           [[-4.8207088 , -4.8221984 , -4.823688  , ..., -7.255672  ,
             -7.2571807 , -7.2585034 ],
            [-4.816544  , -4.8180337 , -4.8195233 , ..., -7.251514  ,
             -7.2530227 , -7.254307  ],
            [-4.812379  , -4.8138685 , -4.815358  , ..., -7.247356  ,
             -7.248864  , -7.2501097 ],
    ...
            [ 3.5294945 ,  3.5279982 ,  3.526502  , ...,  1.0825049 ,
              1.0811739 ,  1.079658  ],
            [ 3.5336335 ,  3.532144  ,  3.530655  , ...,  1.0866923 ,
              1.0853375 ,  1.0838218 ],
            [ 3.5377417 ,  3.5362453 ,  3.5347493 , ...,  1.0908011 ,
              1.0894086 ,  1.0878929 ]],
    
           [[-0.49292195, -0.49441046, -0.49589896, ..., -2.9261274 ,
             -2.9276352 , -2.928957  ],
            [-0.48875785, -0.49024636, -0.4917349 , ..., -2.9219704 ,
             -2.9234781 , -2.9247613 ],
            [-0.48459375, -0.48608226, -0.4875708 , ..., -2.9178133 ,
             -2.919321  , -2.9205654 ],
            ...,
            [ 7.869902  ,  7.868407  ,  7.866911  , ...,  5.423428  ,
              5.4220977 ,  5.420583  ],
            [ 7.8740416 ,  7.8725533 ,  7.871064  , ...,  5.427614  ,
              5.4262605 ,  5.424745  ],
            [ 7.8781505 ,  7.876655  ,  7.8751593 , ...,  5.4317217 ,
              5.4303303 ,  5.428815  ]]], dtype=float32)

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449',
                   '2020-02-24 23:56:51.968040', '2020-03-03 00:03:00.297499',
                   '2020-03-11 23:56:45.657477', '2020-03-19 00:02:52.920718',
                   '2020-03-27 23:56:36.905801'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (2)

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

Open the Data variables (click "‣ Data variables") and click on the stacked cylinders for one of them. You will see the actual data array is available and shown in summary form.

NOTE that you can see real numbers in the array when you do this. This will change when we start using Dask.

This graphical summary of an xarray variable will become increasingly importantly when dask is enabled and as scale out occurs so take a moment now to just poke around the interface. Depending on your area of interest set above, you should have a relatively small area: perhaps around 300 to 400 pixels in each of the x and y dimensions and perhaps up to 10 time slices. This is a relatively small size and fine to do without using Dask.

Next up filter out pixels that are affect by clouds and other issues and compute the NDVI. Since we aren't specifying a time range this will be performed for all images.

%%time
# Identify pixels that don't have cloud, cloud shadow or water
from datacube.utils import masking

cloud_free_mask = masking.make_mask(dataset[qa_band], **qa_mask)

# Apply the mask
cloud_free = dataset.where(cloud_free_mask)

# Calculate the components that make up the NDVI calculation
band_diff = cloud_free[alias['nir']] - cloud_free[alias['red']]
band_sum = cloud_free[alias['nir']] + cloud_free[alias['red']]
# Calculate NDVI
ndvi = None
ndvi = band_diff / band_sum

CPU times: user 841 ms, sys: 1.08 s, total: 1.92 s
Wall time: 1.92 s

The result ndvi is an xarray.DataArray. Let's take a look at it. Again the notebook will render an html version of the data in summary form. Notice again the actual data values are being shown and that there are the same number of time slices as above and the x and y dimensions are identical.

ndvi

<xarray.DataArray (time: 7, y: 2013, x: 1709)>
array([[[       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        ...,
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan]],

       [[0.21968003, 0.21452976, 0.20806599, ..., 0.34342706,
         0.35976648, 0.37791829],
        [0.19766035, 0.19309514, 0.18671527, ..., 0.3560434 ,
         0.36472946, 0.36527237],
        [0.19287958, 0.18996865, 0.18799172, ..., 0.35883703,
         0.36058175, 0.3493558 ],
...
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan],
        [       nan,        nan,        nan, ...,        nan,
                nan,        nan]],

       [[       nan,        nan,        nan, ..., 0.62104238,
         0.66167231, 0.69073634],
        [       nan,        nan,        nan, ..., 0.64705882,
         0.67499382, 0.6999766 ],
        [       nan,        nan,        nan, ..., 0.65460186,
         0.68363546, 0.7074894 ],
        ...,
        [0.66087613, 0.63791103, 0.62416107, ...,        nan,
                nan,        nan],
        [0.66942771, 0.64366472, 0.64570064, ...,        nan,
                nan,        nan],
        [0.61594733, 0.56981982, 0.56342412, ...,        nan,
                nan,        nan]]])
Coordinates:
  * time         (time) datetime64[ns] 2020-02-08T23:56:55.714845 ... 2020-03...
  * y            (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06 -4.049e+06
  * x            (x) float64 1.328e+06 1.328e+06 ... 1.379e+06 1.379e+06
    spatial_ref  int32 3577

xarray.DataArray

• time: 7
• y: 2013
• x: 1709

• nan nan nan nan nan nan nan nan ... nan nan nan nan nan nan nan nan

  array([[[       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          ...,
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan]],
  
         [[0.21968003, 0.21452976, 0.20806599, ..., 0.34342706,
           0.35976648, 0.37791829],
          [0.19766035, 0.19309514, 0.18671527, ..., 0.3560434 ,
           0.36472946, 0.36527237],
          [0.19287958, 0.18996865, 0.18799172, ..., 0.35883703,
           0.36058175, 0.3493558 ],
  ...
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan],
          [       nan,        nan,        nan, ...,        nan,
                  nan,        nan]],
  
         [[       nan,        nan,        nan, ..., 0.62104238,
           0.66167231, 0.69073634],
          [       nan,        nan,        nan, ..., 0.64705882,
           0.67499382, 0.6999766 ],
          [       nan,        nan,        nan, ..., 0.65460186,
           0.68363546, 0.7074894 ],
          ...,
          [0.66087613, 0.63791103, 0.62416107, ...,        nan,
                  nan,        nan],
          [0.66942771, 0.64366472, 0.64570064, ...,        nan,
                  nan,        nan],
          [0.61594733, 0.56981982, 0.56342412, ...,        nan,
                  nan,        nan]]])

• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 ... 2...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000',
           '2020-02-24T23:56:51.968040000', '2020-03-03T00:03:00.297499000',
           '2020-03-11T23:56:45.657477000', '2020-03-19T00:02:52.920718000',
           '2020-03-27T23:56:36.905801000'], dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449',
                   '2020-02-24 23:56:51.968040', '2020-03-03 00:03:00.297499',
                   '2020-03-11 23:56:45.657477', '2020-03-19 00:02:52.920718',
                   '2020-03-27 23:56:36.905801'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (0)

Raw numbers aren't nice to look at so let's draw a time slice. We'll select just one of them to draw and pick one that didn't get masked out by cloud completely. You can see that all clouds and water has been masked out so that we are just looking at the NDVI of the land area.

ndvi.isel(time=1).plot()

<matplotlib.collections.QuadMesh at 0x7fb3eb52dcc0>

Exploring dask concepts with ODC

Let's set our time range to a couple of weeks, or approximately two passes of Landsat 8 for this ROI. Less data will allow us to explore how dask works with the datacube and xarray libraries.

set_time = (set_time[0], parse(set_time[0]) + relativedelta(weeks=3))
# set_time = ("2021-01-01", "2021-01-14")
set_time

('2020-02-01', datetime.datetime(2020, 2, 22, 0, 0))

%%time
dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            group_by=group_by,
        )
dataset

CPU times: user 24.3 s, sys: 3.54 s, total: 27.8 s
Wall time: 51.5 s

<xarray.Dataset>
Dimensions:                     (time: 2, y: 2013, x: 1709)
Coordinates:
  * time                        (time) datetime64[ns] 2020-02-08T23:56:55.714...
  * y                           (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06
  * x                           (x) float64 1.328e+06 1.328e+06 ... 1.379e+06
    spatial_ref                 int32 3577
Data variables: (12/22)
    nbart_coastal_aerosol       (time, y, x) int16 -999 -999 -999 ... 478 499
    nbart_blue                  (time, y, x) int16 -999 -999 -999 ... 491 534
    nbart_green                 (time, y, x) int16 -999 -999 -999 ... 635 682
    nbart_red                   (time, y, x) int16 -999 -999 -999 ... 738 845
    nbart_nir                   (time, y, x) int16 -999 -999 -999 ... 2238 2334
    nbart_swir_1                (time, y, x) int16 -999 -999 -999 ... 1883 2178
    ...                          ...
    oa_relative_slope           (time, y, x) float32 26.82 25.31 ... 141.0 171.0
    oa_satellite_azimuth        (time, y, x) float32 100.1 100.1 ... 280.1 280.1
    oa_satellite_view           (time, y, x) float32 9.022 9.02 ... 7.737 7.74
    oa_solar_azimuth            (time, y, x) float32 67.9 67.9 ... 62.42 62.42
    oa_solar_zenith             (time, y, x) float32 39.49 39.49 ... 39.6 39.6
    oa_time_delta               (time, y, x) float32 -0.4956 -0.4971 ... 1.099
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

• Dimensions:
  • time: 2
  • y: 2013
  • x: 1709
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 2020-...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000'],
          dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Data variables: (22)
  • nbart_coastal_aerosol
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 440 478 499

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1824, 1832, 1861],
            [-999, -999, -999, ..., 1812, 1826, 1858],
            [-999, -999, -999, ..., 1801, 1819, 1846],
            ...,
            [4977, 4976, 4969, ..., 3373, 3424, 3403],
            [4946, 4947, 4948, ..., 3421, 3497, 3461],
            [4918, 4947, 4946, ..., 3507, 3553, 3487]],
    
           [[ 868,  860,  862, ...,  644,  627,  630],
            [ 827,  857,  857, ...,  615,  618,  648],
            [ 815,  831,  838, ...,  608,  624,  651],
            ...,
            [ 468,  350,  327, ...,  642,  644,  623],
            [ 394,  328,  310, ...,  569,  561,  545],
            [ 353,  327,  311, ...,  440,  478,  499]]], dtype=int16)

  • nbart_blue
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 482 491 534

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1854, 1868, 1889],
            [-999, -999, -999, ..., 1839, 1858, 1885],
            [-999, -999, -999, ..., 1833, 1851, 1881],
            ...,
            [4951, 4963, 4941, ..., 3358, 3411, 3387],
            [4919, 4932, 4933, ..., 3408, 3481, 3444],
            [4901, 4937, 4937, ..., 3488, 3543, 3470]],
    
           [[1074, 1070, 1070, ...,  764,  740,  739],
            [ 999, 1027, 1019, ...,  729,  732,  759],
            [ 978,  995, 1005, ...,  718,  739,  769],
            ...,
            [ 467,  322,  296, ...,  776,  753,  716],
            [ 376,  289,  278, ...,  641,  640,  615],
            [ 324,  287,  283, ...,  482,  491,  534]]], dtype=int16)

  • nbart_green
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 569 635 682

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1971, 1982, 2003],
            [-999, -999, -999, ..., 1950, 1963, 1993],
            [-999, -999, -999, ..., 1942, 1958, 1985],
            ...,
            [4932, 4943, 4925, ..., 3364, 3423, 3400],
            [4903, 4922, 4912, ..., 3417, 3488, 3455],
            [4890, 4923, 4912, ..., 3505, 3558, 3483]],
    
           [[1635, 1584, 1574, ..., 1074, 1040, 1030],
            [1458, 1479, 1466, ..., 1011, 1015, 1047],
            [1416, 1429, 1445, ..., 1000, 1023, 1047],
            ...,
            [ 486,  335,  295, ..., 1047, 1003,  940],
            [ 405,  292,  275, ...,  833,  811,  790],
            [ 349,  294,  285, ...,  569,  635,  682]]], dtype=int16)

  • nbart_red
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 665 738 845

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2137, 2148, 2168],
            [-999, -999, -999, ..., 2111, 2122, 2158],
            [-999, -999, -999, ..., 2103, 2123, 2150],
            ...,
            [4912, 4938, 4919, ..., 3382, 3447, 3421],
            [4883, 4911, 4901, ..., 3446, 3527, 3483],
            [4880, 4915, 4905, ..., 3538, 3596, 3514]],
    
           [[2268, 2184, 2160, ..., 1366, 1316, 1279],
            [1989, 2010, 2008, ..., 1276, 1268, 1305],
            [1927, 1938, 1961, ..., 1257, 1275, 1313],
            ...,
            [ 486,  328,  294, ..., 1339, 1273, 1198],
            [ 386,  274,  252, ..., 1042, 1016,  959],
            [ 332,  279,  273, ...,  665,  738,  845]]], dtype=int16)

  • nbart_nir
    (time, y, x)
    int16
    -999 -999 -999 ... 2082 2238 2334

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2823, 2837, 2861],
            [-999, -999, -999, ..., 2797, 2813, 2845],
            [-999, -999, -999, ..., 2788, 2806, 2829],
            ...,
            [5169, 5179, 5158, ..., 3927, 3985, 3957],
            [5143, 5160, 5150, ..., 3988, 4065, 4018],
            [5133, 5160, 5150, ..., 4078, 4127, 4049]],
    
           [[3545, 3377, 3295, ..., 2795, 2795, 2833],
            [2969, 2972, 2930, ..., 2687, 2724, 2807],
            [2848, 2847, 2869, ..., 2664, 2713, 2723],
            ...,
            [1514, 1132,  987, ..., 2529, 2356, 2150],
            [1309,  915,  867, ..., 2271, 2186, 2169],
            [1173,  820,  789, ..., 2082, 2238, 2334]]], dtype=int16)

  • nbart_swir_1
    (time, y, x)
    int16
    -999 -999 -999 ... 1830 1883 2178

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1958, 1966, 1988],
            [-999, -999, -999, ..., 1947, 1956, 1983],
            [-999, -999, -999, ..., 1947, 1952, 1978],
            ...,
            [1747, 1758, 1744, ..., 1696, 1724, 1709],
            [1730, 1733, 1716, ..., 1726, 1766, 1743],
            [1702, 1707, 1687, ..., 1780, 1803, 1761]],
    
           [[4288, 4196, 4151, ..., 3679, 3685, 3739],
            [3848, 3908, 3925, ..., 3643, 3688, 3780],
            [3768, 3813, 3860, ..., 3667, 3685, 3769],
            ...,
            [ 971,  695,  588, ..., 3403, 3494, 3418],
            [ 769,  525,  457, ..., 2754, 3002, 2980],
            [ 638,  478,  447, ..., 1830, 1883, 2178]]], dtype=int16)

  • nbart_swir_2
    (time, y, x)
    int16
    -999 -999 -999 ... 990 1184 1331

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1590, 1599, 1617],
            [-999, -999, -999, ..., 1583, 1592, 1618],
            [-999, -999, -999, ..., 1585, 1583, 1610],
            ...,
            [1933, 1938, 1925, ..., 1643, 1677, 1664],
            [1904, 1912, 1901, ..., 1677, 1721, 1696],
            [1878, 1894, 1881, ..., 1735, 1763, 1718]],
    
           [[3109, 3091, 3087, ..., 2571, 2546, 2554],
            [3010, 3036, 3074, ..., 2537, 2549, 2634],
            [2940, 2991, 3045, ..., 2555, 2563, 2686],
            ...,
            [ 541,  351,  305, ..., 2207, 2358, 2308],
            [ 397,  254,  226, ..., 1788, 1917, 1954],
            [ 319,  247,  244, ...,  990, 1184, 1331]]], dtype=int16)

  • nbart_panchromatic
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 583 598 621

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2007, 2012, 2038],
            [-999, -999, -999, ..., 1968, 1988, 2034],
            [-999, -999, -999, ..., 1970, 1982, 2029],
            ...,
            [4886, 4887, 4879, ..., 3382, 3442, 3396],
            [4841, 4846, 4866, ..., 3439, 3529, 3472],
            [4823, 4857, 4853, ..., 3524, 3604, 3520]],
    
           [[1965, 1849, 1795, ..., 1170, 1085, 1095],
            [1595, 1604, 1611, ..., 1049, 1066, 1106],
            [1574, 1555, 1602, ..., 1066, 1068, 1120],
            ...,
            [ 553,  298,  273, ..., 1185, 1093,  995],
            [ 362,  253,  228, ...,  925,  919,  881],
            [ 321,  266,  273, ...,  583,  598,  621]]], dtype=int16)

  • oa_fmask
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    1

    nodata :

    0

    flags_definition :

    {'fmask': {'bits': [0, 1, 2, 3, 4, 5, 6, 7], 'values': {'0': 'nodata', '1': 'valid', '2': 'cloud', '3': 'shadow', '4': 'snow', '5': 'water'}, 'description': 'Fmask'}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            ...,
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0]]], dtype=uint8)

  • oa_nbart_contiguity
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    flags_definition :

    {'contiguous': {'bits': [0], 'values': {'0': False, '1': True}}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_azimuthal_exiting
    (time, y, x)
    float32
    -80.12 -69.12 ... -73.24 -37.26

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -80.11553  ,  -69.122116 ,  -74.337006 , ...,   33.770046 ,
               97.201294 ,  133.90964  ],
            [ -98.09167  ,  -94.05396  ,  -87.45524  , ..., -135.61787  ,
              -76.76864  ,   76.9891   ],
            [-102.43699  , -115.73346  , -107.09284  , ..., -145.66154  ,
             -141.03618  , -102.94009  ],
            ...,
            [  18.39366  ,   13.645186 ,   17.93953  , ...,   60.979168 ,
               83.76587  ,   66.93567  ],
            [  19.606314 ,   11.669298 ,   11.353658 , ...,   88.5983   ,
               61.46648  ,   47.17998  ],
            [  11.797114 ,    2.8835816,    3.9532704, ...,   91.169556 ,
               22.845282 ,   38.14933  ]],
    
           [[  68.512245 ,   70.72293  ,   63.153164 , ...,  -57.384872 ,
              -57.144466 ,   -8.937615 ],
            [  57.491165 ,   53.669292 ,   51.882816 , ...,   62.990677 ,
               32.850204 , -101.98451  ],
            [  41.615105 ,   28.149704 ,   29.797287 , ...,   56.238945 ,
               50.34515  ,  -35.11987  ],
            ...,
            [ -55.327877 ,  -60.8111   ,  -58.213642 , ...,   42.198055 ,
               32.045265 ,   14.97294  ],
            [ -56.430367 ,  -58.39121  ,  -54.94616  , ...,   36.177956 ,
              -21.378311 ,  -20.307953 ],
            [ -52.08491  ,  -50.880047 ,  -45.48003  , ...,  -16.137846 ,
              -73.24443  ,  -37.25841  ]]], dtype=float32)

  • oa_azimuthal_incident
    (time, y, x)
    float32
    -53.29 -43.81 ... 101.3 106.9

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -53.29337 ,  -43.814816,  -49.677975, ...,   73.61261 ,
               83.28998 , -150.34354 ],
            [ -70.001015,  -67.43494 ,  -62.690357, ...,  -89.084045,
              -35.280712,  110.770065],
            [ -77.07651 ,  -90.91911 ,  -84.01256 , ...,  -97.06677 ,
             -102.89715 ,  -62.93791 ],
            ...,
            [  71.86952 ,   65.37238 ,   70.617676, ...,  -37.93936 ,
             -127.73148 ,   55.0821  ],
            [  72.86513 ,   63.776108,   64.33563 , ...,  -86.956635,
               92.59873 ,  138.75403 ],
            [  65.48859 ,   55.617256,   58.376175, ..., -145.5778  ,
               96.301636,  128.84294 ]],
    
           [[ -49.41655 ,  -40.01738 ,  -45.558876, ...,   77.89437 ,
               87.38188 , -146.33037 ],
            [ -65.91871 ,  -64.03974 ,  -59.186226, ...,  -84.63618 ,
              -31.004824,  115.21529 ],
            [ -72.751   ,  -87.00204 ,  -80.26874 , ...,  -92.55007 ,
              -98.49117 ,  -58.568077],
            ...,
            [  76.22296 ,   69.71172 ,   75.02115 , ...,  -32.531303,
             -121.95462 ,   60.88784 ],
            [  77.265816,   68.14368 ,   68.7059  , ...,  -81.56631 ,
               98.153   ,  143.99895 ],
            [  69.8667  ,   59.935596,   62.67588 , ..., -138.76768 ,
              101.29737 ,  106.94179 ]]], dtype=float32)

  • oa_combined_terrain_shadow
    (time, y, x)
    uint8
    1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_exiting_angle
    (time, y, x)
    float32
    9.029 9.148 9.09 ... 8.081 10.43

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 9.028952 ,  9.148497 ,  9.090177 , ...,  3.2647538,
              3.62325  ,  4.391917 ],
            [ 8.894047 ,  8.901706 ,  8.9225235, ...,  3.2806778,
              3.5062037,  4.4214954],
            [ 8.756343 ,  8.535571 ,  8.580602 , ...,  3.3433805,
              3.560637 ,  4.636894 ],
            ...,
            [11.375915 , 11.375726 , 11.181578 , ...,  5.0770392,
              4.6290197,  6.9061594],
            [11.1641655, 11.834425 , 12.261691 , ...,  3.2524686,
              5.314759 ,  7.5270324],
            [12.537902 , 13.934173 , 14.685269 , ...,  3.791945 ,
              6.8892117,  7.5075154]],
    
           [[ 2.4255953,  2.4921818,  2.615967 , ...,  7.997434 ,
              7.613728 ,  6.8861303],
            [ 2.4387937,  2.5729144,  2.7521896, ...,  8.068149 ,
              7.777022 ,  6.803534 ],
            [ 2.8313277,  3.079027 ,  3.2421515, ...,  8.0514555,
              7.821932 ,  6.652463 ],
            ...,
            [ 5.45706  ,  5.047012 ,  5.174448 , ..., 15.872423 ,
             13.216334 , 13.8058195],
            [ 5.292245 ,  5.412933 ,  5.899394 , ..., 12.911547 ,
             10.252516 , 11.655185 ],
            [ 6.263888 ,  7.0900445,  8.104066 , ...,  9.325176 ,
              8.080662 , 10.431503 ]]], dtype=float32)

  • oa_incident_angle
    (time, y, x)
    float32
    40.0 40.23 40.24 ... 37.89 35.61

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.998978, 40.23141 , 40.237175, ..., 37.26962 , 37.826603,
             38.7017  ],
            [39.77791 , 39.912277, 40.08276 , ..., 37.007404, 37.392212,
             38.432606],
            [39.88898 , 39.73803 , 39.905514, ..., 36.956375, 37.170578,
             38.289707],
            ...,
            [39.404842, 39.735092, 39.47829 , ..., 31.522882, 33.095287,
             32.230022],
            [39.361443, 39.853664, 39.843723, ..., 33.740055, 35.684074,
             34.42446 ],
            [39.81281 , 40.565845, 40.41699 , ..., 36.779434, 37.866028,
             35.635822]],
    
           [[40.41124 , 40.656933, 40.68124 , ..., 37.69816 , 38.263546,
             39.134884],
            [40.19523 , 40.337387, 40.529827, ..., 37.413853, 37.801704,
             38.827084],
            [40.35013 , 40.206455, 40.390034, ..., 37.354183, 37.558178,
             38.643627],
            ...,
            [39.349937, 39.697414, 39.442852, ..., 32.172   , 33.430336,
             32.340042],
            [39.322533, 39.776737, 39.72588 , ..., 34.17244 , 35.808483,
             34.376877],
            [39.66718 , 40.32737 , 40.102894, ..., 37.021255, 37.88646 ,
             35.605076]]], dtype=float32)

  • oa_relative_azimuth
    (time, y, x)
    float32
    32.23 32.23 32.23 ... 217.7 217.7

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 32.232582,  32.23285 ,  32.233112, ...,  32.693703,
              32.69397 ,  32.69425 ],
            [ 32.23295 ,  32.23321 ,  32.233475, ...,  32.69407 ,
              32.69434 ,  32.694626],
            [ 32.233315,  32.233578,  32.23384 , ...,  32.694443,
              32.694706,  32.694992],
            ...,
            [ 32.975964,  32.976234,  32.976505, ...,  33.447357,
              33.447643,  33.447918],
            [ 32.976337,  32.976612,  32.976875, ...,  33.447735,
              33.448017,  33.44829 ],
            [ 32.976727,  32.976997,  32.977264, ...,  33.44813 ,
              33.448418,  33.44869 ]],
    
           [[216.49684 , 216.49713 , 216.49739 , ..., 216.99608 ,
             216.99635 , 216.99666 ],
            [216.4972  , 216.49747 , 216.49776 , ..., 216.99643 ,
             216.99673 , 216.99702 ],
            [216.49756 , 216.49783 , 216.49811 , ..., 216.9968  ,
             216.99709 , 216.99738 ],
            ...,
            [217.21596 , 217.21623 , 217.21652 , ..., 217.72452 ,
             217.72482 , 217.72513 ],
            [217.21632 , 217.21661 , 217.21689 , ..., 217.72487 ,
             217.72519 , 217.7255  ],
            [217.2167  , 217.217   , 217.21727 , ..., 217.72527 ,
             217.72557 , 217.72588 ]]], dtype=float32)

  • oa_relative_slope
    (time, y, x)
    float32
    26.82 25.31 24.66 ... 141.0 171.0

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[  26.822166,   25.307304,   24.659027, ...,   39.842567,
               32.689396,   26.666718],
            [  28.09065 ,   26.619028,   24.764885, ...,   46.53382 ,
               41.487926,   33.780968],
            [  25.360477,   24.81435 ,   23.08028 , ...,   48.594772,
               47.248375,   40.002182],
            ...,
            [  53.47586 ,   51.727196,   52.678143, ..., -169.05045 ,
               43.374695,  110.16484 ],
            [  53.258812,   52.106808,   52.98198 , ...,  116.92698 ,
               94.31888 ,  100.24516 ],
            [  53.69147 ,   52.733677,   54.422905, ...,   82.8719  ,
               73.45635 ,   90.69361 ]],
    
           [[-117.928795, -110.74031 , -108.71204 , ..., -144.39311 ,
             -141.10318 , -137.39276 ],
            [-123.409874, -117.70903 , -111.069046, ..., -147.62685 ,
             -145.64392 , -142.8002  ],
            [-114.366104, -115.15174 , -110.066025, ..., -148.78902 ,
             -148.83632 , -148.09137 ],
            ...,
            [ 131.55084 ,  130.52283 ,  133.23479 , ..., -144.8613  ,
             -160.54012 ,   45.9149  ],
            [ 133.69618 ,  126.53488 ,  123.652054, ..., -152.83937 ,
             -173.85126 ,  135.26845 ],
            [ 121.951614,  110.81564 ,  108.15591 , ..., -122.62984 ,
              140.97137 ,  170.97281 ]]], dtype=float32)

  • oa_satellite_azimuth
    (time, y, x)
    float32
    100.1 100.1 100.1 ... 280.1 280.1

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[100.1332  , 100.13319 , 100.13318 , ..., 100.11447 ,
             100.114456, 100.11445 ],
            [100.13323 , 100.13322 , 100.13321 , ..., 100.1145  ,
             100.114494, 100.11449 ],
            [100.13326 , 100.13325 , 100.13324 , ..., 100.11453 ,
             100.114525, 100.11452 ],
            ...,
            [100.19855 , 100.19853 , 100.198524, ..., 100.179245,
             100.17924 , 100.17922 ],
            [100.19858 , 100.19857 , 100.198555, ..., 100.17928 ,
             100.179276, 100.17926 ],
            [100.198616, 100.1986  , 100.198586, ..., 100.17931 ,
             100.1793  , 100.17929 ]],
    
           [[280.09695 , 280.09692 , 280.09692 , ..., 280.07846 ,
             280.07843 , 280.07843 ],
            [280.097   , 280.09695 , 280.09695 , ..., 280.0785  ,
             280.0785  , 280.07846 ],
            [280.09702 , 280.09702 , 280.097   , ..., 280.07852 ,
             280.07852 , 280.0785  ],
            ...,
            [280.16147 , 280.16144 , 280.1614  , ..., 280.1424  ,
             280.1424  , 280.1424  ],
            [280.1615  , 280.16147 , 280.16144 , ..., 280.14243 ,
             280.14243 , 280.14243 ],
            [280.16153 , 280.1615  , 280.16147 , ..., 280.14246 ,
             280.14246 , 280.14246 ]]], dtype=float32)

  • oa_satellite_view
    (time, y, x)
    float32
    9.022 9.02 9.017 ... 7.737 7.74

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[9.02193  , 9.019681 , 9.017433 , ..., 5.130571 , 5.1282773,
             5.125959 ],
            [9.021114 , 9.018867 , 9.016618 , ..., 5.1297407, 5.1274476,
             5.1251235],
            [9.0203   , 9.018051 , 9.015802 , ..., 5.128911 , 5.1266174,
             5.1242886],
            ...,
            [7.31853  , 7.3162613, 7.3139925, ..., 3.4020782, 3.3997495,
             3.3974464],
            [7.31763  , 7.3153787, 7.3131275, ..., 3.401222 , 3.3989155,
             3.3966124],
            [7.316651 , 7.314382 , 7.3121133, ..., 3.400168 , 3.397848 ,
             3.3955448]],
    
           [[2.1344194, 2.1367428, 2.1390662, ..., 6.0918517, 6.094148 ,
             6.096469 ],
            [2.1352227, 2.137546 , 2.1398695, ..., 6.092644 , 6.094941 ,
             6.097266 ],
            [2.1360261, 2.1383495, 2.140673 , ..., 6.0934367, 6.095733 ,
             6.0980635],
            ...,
            [3.803363 , 3.805676 , 3.8079891, ..., 7.733246 , 7.7355447,
             7.7378206],
            [3.804241 , 3.806536 , 3.808831 , ..., 7.734052 , 7.7363305,
             7.738606 ],
            [3.8051991, 3.8075123, 3.8098252, ..., 7.7350545, 7.737346 ,
             7.7396216]]], dtype=float32)

  • oa_solar_azimuth
    (time, y, x)
    float32
    67.9 67.9 67.9 ... 62.42 62.42

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[67.90062 , 67.900345, 67.90006 , ..., 67.42077 , 67.42049 ,
             67.4202  ],
            [67.90028 , 67.90001 , 67.899734, ..., 67.42043 , 67.42015 ,
             67.41986 ],
            [67.89995 , 67.89967 , 67.8994  , ..., 67.4201  , 67.419815,
             67.41952 ],
            ...,
            [67.22258 , 67.2223  , 67.222015, ..., 66.73189 , 66.73159 ,
             66.73131 ],
            [67.222244, 67.221954, 67.22168 , ..., 66.73155 , 66.731255,
             66.73097 ],
            [67.221886, 67.2216  , 67.22132 , ..., 66.73118 , 66.73089 ,
             66.7306  ]],
    
           [[63.60011 , 63.59981 , 63.599514, ..., 63.082386, 63.082085,
             63.08177 ],
            [63.59979 , 63.59949 , 63.599194, ..., 63.082058, 63.081757,
             63.081444],
            [63.599464, 63.599167, 63.59887 , ..., 63.08173 , 63.08143 ,
             63.081112],
            ...,
            [62.94549 , 62.94519 , 62.944885, ..., 62.41788 , 62.41756 ,
             62.417255],
            [62.945164, 62.94486 , 62.94456 , ..., 62.417553, 62.417236,
             62.41693 ],
            [62.94482 , 62.944515, 62.944214, ..., 62.417194, 62.416878,
             62.416573]]], dtype=float32)

  • oa_solar_zenith
    (time, y, x)
    float32
    39.49 39.49 39.49 ... 39.6 39.6

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.487972, 39.48771 , 39.48745 , ..., 39.04551 , 39.04525 ,
             39.044994],
            [39.48805 , 39.487785, 39.487526, ..., 39.045586, 39.045326,
             39.04507 ],
            [39.48812 , 39.48786 , 39.4876  , ..., 39.045662, 39.045403,
             39.045147],
            ...,
            [39.63633 , 39.63607 , 39.63581 , ..., 39.195976, 39.195724,
             39.19546 ],
            [39.636402, 39.636143, 39.635883, ..., 39.196053, 39.1958  ,
             39.19554 ],
            [39.636463, 39.636204, 39.635944, ..., 39.196114, 39.195854,
             39.195595]],
    
           [[39.84592 , 39.845665, 39.845406, ..., 39.414013, 39.413757,
             39.413506],
            [39.846012, 39.845757, 39.8455  , ..., 39.41411 , 39.413853,
             39.413605],
            [39.846107, 39.84585 , 39.845596, ..., 39.414204, 39.413948,
             39.4137  ],
            ...,
            [40.032944, 40.032692, 40.032436, ..., 39.60342 , 39.603172,
             39.602917],
            [40.033035, 40.032784, 40.03253 , ..., 39.603516, 39.60327 ,
             39.603016],
            [40.033115, 40.032864, 40.032608, ..., 39.603596, 39.603344,
             39.603092]]], dtype=float32)

  • oa_time_delta
    (time, y, x)
    float32
    -0.4956 -0.4971 ... 1.101 1.099

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-0.49559563, -0.4970854 , -0.49857512, ..., -2.9306152 ,
             -2.9321241 , -2.9334474 ],
            [-0.49143195, -0.49292168, -0.49441144, ..., -2.9264584 ,
             -2.9279675 , -2.929252  ],
            [-0.48726824, -0.48875797, -0.49024773, ..., -2.9223018 ,
             -2.923811  , -2.9250565 ],
            ...,
            [ 7.86643   ,  7.864933  ,  7.863436  , ...,  5.4181333 ,
              5.416802  ,  5.4152856 ],
            [ 7.870569  ,  7.869079  ,  7.867589  , ...,  5.422319  ,
              5.420964  ,  5.4194474 ],
            [ 7.874677  ,  7.8731804 ,  7.8716836 , ...,  5.4264264 ,
              5.4250336 ,  5.423517  ]],
    
           [[-4.8207088 , -4.8221984 , -4.823688  , ..., -7.255672  ,
             -7.2571807 , -7.2585034 ],
            [-4.816544  , -4.8180337 , -4.8195233 , ..., -7.251514  ,
             -7.2530227 , -7.254307  ],
            [-4.812379  , -4.8138685 , -4.815358  , ..., -7.247356  ,
             -7.248864  , -7.2501097 ],
            ...,
            [ 3.5420945 ,  3.5405984 ,  3.539102  , ...,  1.0940956 ,
              1.0927645 ,  1.0912488 ],
            [ 3.5462334 ,  3.5447443 ,  3.543255  , ...,  1.0982829 ,
              1.0969281 ,  1.0954124 ],
            [ 3.5503414 ,  3.5488453 ,  3.547349  , ...,  1.1023915 ,
              1.1009991 ,  1.0994833 ]]], dtype=float32)

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449'], dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (2)

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

As before you can see the actual data in the results but this time there should only be 1 or 2 observation times

Start a dask LocalCluster

Now let's create a LocalCluster as we did in the earlier notebook.

from dask.distributed import Client, LocalCluster

cluster = LocalCluster()
client = Client(cluster)
client

Client

Client-6b773375-0464-11ee-926b-06a3e810ea18

Connection method: Cluster object	Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Launch dashboard in JupyterLab

Cluster Info

LocalCluster

2558b75e

Dashboard: http://127.0.0.1:8787/status	Workers: 4
Total threads: 8	Total memory: 24.00 GiB
Status: running	Using processes: True

Scheduler Info

Scheduler

Scheduler-9d67f45d-1c7d-4c94-8ea4-b0b4484205a4

Comm: tcp://127.0.0.1:45489	Workers: 4
Dashboard: http://127.0.0.1:8787/status	Total threads: 8
Started: Just now	Total memory: 24.00 GiB

Workers

Worker: 0

Comm: tcp://127.0.0.1:33385	Total threads: 2
Dashboard: http://127.0.0.1:45065/status	Memory: 6.00 GiB
Nanny: tcp://127.0.0.1:43585
Local directory: /tmp/dask-worker-space/worker-81no5071

Worker: 1

Comm: tcp://127.0.0.1:35215	Total threads: 2
Dashboard: http://127.0.0.1:42473/status	Memory: 6.00 GiB
Nanny: tcp://127.0.0.1:46689
Local directory: /tmp/dask-worker-space/worker-0z3hhmct

Worker: 2

Comm: tcp://127.0.0.1:39851	Total threads: 2
Dashboard: http://127.0.0.1:41123/status	Memory: 6.00 GiB
Nanny: tcp://127.0.0.1:46171
Local directory: /tmp/dask-worker-space/worker-1x8q2iow

Worker: 3

Comm: tcp://127.0.0.1:43485	Total threads: 2
Dashboard: http://127.0.0.1:39837/status	Memory: 6.00 GiB
Nanny: tcp://127.0.0.1:43977
Local directory: /tmp/dask-worker-space/worker-st0xxu0o

You may like to open up the dashboard for the cluster, although for this notebook we won't be talking about the dashboard (that's for a later discussion).

notebook_utils.localcluster_dashboard(client=client, server=easi.hub)

'https://hub.csiro.easi-eo.solutions/user/csiro-csiro-aad_pag064@csiro.au/proxy/8787/status'

Now that we are using a cluster, even though it is local, we need to make sure that our cluster has the right configuration to use Requester Pays buckets in AWS S3. To do this, we need to re-run the configure_s3_access() function that we ran earlier, but we need to pass the client to the function as well.

from datacube.utils.aws import configure_s3_access
configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);

datacube.load() will use the default dask cluster (the one we just created) if the dask_chunks parameter is specified.

The chunk shape and memory size is a critial parameter in tuning dask and we will be discussing it in great detail as scale increases. For now we're simply going to specify that the time dimension should individually chunked (1 slice of time) and by not specifying any chunking for the other dimensions they will be form a single contiguous block.

If that made no sense what's so ever, that's fine because we will look at an example.

chunks = {"time":1}

%%time
dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, ###### THIS IS THE ONLY LINE CHANGED. #####
            group_by=group_by,
        )
dataset

CPU times: user 184 ms, sys: 10.4 ms, total: 194 ms
Wall time: 230 ms

<xarray.Dataset>
Dimensions:                     (time: 2, y: 2013, x: 1709)
Coordinates:
  * time                        (time) datetime64[ns] 2020-02-08T23:56:55.714...
  * y                           (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06
  * x                           (x) float64 1.328e+06 1.328e+06 ... 1.379e+06
    spatial_ref                 int32 3577
Data variables: (12/22)
    nbart_coastal_aerosol       (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_blue                  (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_green                 (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_red                   (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_nir                   (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_swir_1                (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    ...                          ...
    oa_relative_slope           (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_satellite_azimuth        (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_satellite_view           (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_solar_azimuth            (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_solar_zenith             (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_time_delta               (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

• Dimensions:
  • time: 2
  • y: 2013
  • x: 1709
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 2020-...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000'],
          dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Data variables: (22)
  • nbart_coastal_aerosol
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_blue
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_green
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_red
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_nir
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_swir_1
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_swir_2
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_panchromatic
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 13.12 MiB 6.56 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type int16 numpy.ndarray

  • oa_fmask
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    0

    flags_definition :

    {'fmask': {'bits': [0, 1, 2, 3, 4, 5, 6, 7], 'values': {'0': 'nodata', '1': 'valid', '2': 'cloud', '3': 'shadow', '4': 'snow', '5': 'water'}, 'description': 'Fmask'}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 6.56 MiB 3.28 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type uint8 numpy.ndarray

  • oa_nbart_contiguity
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    255

    flags_definition :

    {'contiguous': {'bits': [0], 'values': {'0': False, '1': True}}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 6.56 MiB 3.28 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type uint8 numpy.ndarray

  • oa_azimuthal_exiting
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_azimuthal_incident
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_combined_terrain_shadow
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    255

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 6.56 MiB 3.28 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type uint8 numpy.ndarray

  • oa_exiting_angle
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_incident_angle
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_relative_azimuth
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_relative_slope
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_satellite_azimuth
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_satellite_view
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_solar_azimuth
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_solar_zenith
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_time_delta
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 26.25 MiB 13.12 MiB Shape (2, 2013, 1709) (1, 2013, 1709) Dask graph 2 chunks in 1 graph layer Data type float32 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449'], dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (2)

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

First thing you probably noticed is that whilst only one line changed the load time dropped to sub-seconds! The second thing you probably noticed is if you look at one of the data variables by clicking on the database icon as before, there is no data but instead there is a diagram which shows you the Dask Chunks for each measurement. It's really fast because it didn't actually load any data!

When datatcube has dask_chunks specified it switches from creating xarrays to instead use dask.arrays in the backend and lazy loads them - this means that no data is loaded until used. If you look at one of the data variables you will see it now has dask.array<chunksize=(....)> rather than values and the cylinder icon will show the Array and Chunk parameters along with some statistics, not actual data.

The datacube.load() has used the dask.Delayed interface which will not perform any tasks (Dask's name for calculations) until the result of the task is actually required. We'll load the data in a moment but first let's take a look at the parameters in that pretty visualisation. Click on the cylinder for the red Data variables and look at the table and the figure. It should look similar to the image below.

Looking at this image (yours may be different), you can see that:

1. The Array is 221.92 kiB in total size and is broken into Chunks which have size 110.96 kiB
2. The Array shape is (2, 375, 303) (time, y, x) but each chunk is (1,375,303) because we specified the time dimension should have chunks of length 1.
3. There are 2 chunk tasks, one for each time slice and in this instance, and one graph layer. More complex calculations will have more layers in the graph.
4. The Array type is uint16 and is split up into chunks which are numpy.ndarrays.

The chunking has split the array loading into two Chunks. Dask can execute these in parallel.

We can look at the delayed tasks and how they will be executed by visualising the task graph for one of the variables. We'll use the red band measurement.

dataset[alias['red']].data.visualize()

Details on the task graph can be found in the dask user guide but what's clear is you have two independent paths of execution which produce one time slice each (0,0,0) and (1,0,0). These are the two chunks that that full array has been split into.

To retrieve the actual data we need to compute() the result, this will cause all the delayed tasks to be executed for the variable we are computing. Let's compute() the red variable.

%%time
actual_red = dataset[alias['red']].compute()
actual_red

CPU times: user 87.4 ms, sys: 29.3 ms, total: 117 ms
Wall time: 2.02 s

<xarray.DataArray 'nbart_red' (time: 2, y: 2013, x: 1709)>
array([[[-999, -999, -999, ..., 2137, 2148, 2168],
        [-999, -999, -999, ..., 2111, 2122, 2158],
        [-999, -999, -999, ..., 2103, 2123, 2150],
        ...,
        [4912, 4938, 4919, ..., 3382, 3447, 3421],
        [4883, 4911, 4901, ..., 3446, 3527, 3483],
        [4880, 4915, 4905, ..., 3538, 3596, 3514]],

       [[2268, 2184, 2160, ..., 1366, 1316, 1279],
        [1989, 2010, 2008, ..., 1276, 1268, 1305],
        [1927, 1938, 1961, ..., 1257, 1275, 1313],
        ...,
        [ 486,  328,  294, ..., 1339, 1273, 1198],
        [ 386,  274,  252, ..., 1042, 1016,  959],
        [ 332,  279,  273, ...,  665,  738,  845]]], dtype=int16)
Coordinates:
  * time         (time) datetime64[ns] 2020-02-08T23:56:55.714845 2020-02-16T...
  * y            (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06 -4.049e+06
  * x            (x) float64 1.328e+06 1.328e+06 ... 1.379e+06 1.379e+06
    spatial_ref  int32 3577
Attributes:
    units:         1
    nodata:        -999
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.DataArray

'nbart_red'

• time: 2
• y: 2013
• x: 1709

• -999 -999 -999 -999 -999 -999 -999 ... 808 820 768 718 665 738 845

  array([[[-999, -999, -999, ..., 2137, 2148, 2168],
          [-999, -999, -999, ..., 2111, 2122, 2158],
          [-999, -999, -999, ..., 2103, 2123, 2150],
          ...,
          [4912, 4938, 4919, ..., 3382, 3447, 3421],
          [4883, 4911, 4901, ..., 3446, 3527, 3483],
          [4880, 4915, 4905, ..., 3538, 3596, 3514]],
  
         [[2268, 2184, 2160, ..., 1366, 1316, 1279],
          [1989, 2010, 2008, ..., 1276, 1268, 1305],
          [1927, 1938, 1961, ..., 1257, 1275, 1313],
          ...,
          [ 486,  328,  294, ..., 1339, 1273, 1198],
          [ 386,  274,  252, ..., 1042, 1016,  959],
          [ 332,  279,  273, ...,  665,  738,  845]]], dtype=int16)

• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 2020-...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000'],
          dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449'], dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (4)

  units :

  1

  nodata :

  -999

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

As you can see we now have actual data (there are real numbers, not just Dask arrays). You can do the same thing for all arrays in the dataset in one go by computing the dataset itself.

%%time
actual_dataset = dataset.compute()
actual_dataset

CPU times: user 596 ms, sys: 430 ms, total: 1.03 s
Wall time: 9.5 s

<xarray.Dataset>
Dimensions:                     (time: 2, y: 2013, x: 1709)
Coordinates:
  * time                        (time) datetime64[ns] 2020-02-08T23:56:55.714...
  * y                           (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06
  * x                           (x) float64 1.328e+06 1.328e+06 ... 1.379e+06
    spatial_ref                 int32 3577
Data variables: (12/22)
    nbart_coastal_aerosol       (time, y, x) int16 -999 -999 -999 ... 478 499
    nbart_blue                  (time, y, x) int16 -999 -999 -999 ... 491 534
    nbart_green                 (time, y, x) int16 -999 -999 -999 ... 635 682
    nbart_red                   (time, y, x) int16 -999 -999 -999 ... 738 845
    nbart_nir                   (time, y, x) int16 -999 -999 -999 ... 2238 2334
    nbart_swir_1                (time, y, x) int16 -999 -999 -999 ... 1883 2178
    ...                          ...
    oa_relative_slope           (time, y, x) float32 26.82 25.31 ... 141.0 171.0
    oa_satellite_azimuth        (time, y, x) float32 100.1 100.1 ... 280.1 280.1
    oa_satellite_view           (time, y, x) float32 9.022 9.02 ... 7.737 7.74
    oa_solar_azimuth            (time, y, x) float32 67.9 67.9 ... 62.42 62.42
    oa_solar_zenith             (time, y, x) float32 39.49 39.49 ... 39.6 39.6
    oa_time_delta               (time, y, x) float32 -0.4956 -0.4971 ... 1.099
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

• Dimensions:
  • time: 2
  • y: 2013
  • x: 1709
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 2020-...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000'],
          dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Data variables: (22)
  • nbart_coastal_aerosol
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 440 478 499

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1824, 1832, 1861],
            [-999, -999, -999, ..., 1812, 1826, 1858],
            [-999, -999, -999, ..., 1801, 1819, 1846],
            ...,
            [4977, 4976, 4969, ..., 3373, 3424, 3403],
            [4946, 4947, 4948, ..., 3421, 3497, 3461],
            [4918, 4947, 4946, ..., 3507, 3553, 3487]],
    
           [[ 868,  860,  862, ...,  644,  627,  630],
            [ 827,  857,  857, ...,  615,  618,  648],
            [ 815,  831,  838, ...,  608,  624,  651],
            ...,
            [ 468,  350,  327, ...,  642,  644,  623],
            [ 394,  328,  310, ...,  569,  561,  545],
            [ 353,  327,  311, ...,  440,  478,  499]]], dtype=int16)

  • nbart_blue
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 482 491 534

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1854, 1868, 1889],
            [-999, -999, -999, ..., 1839, 1858, 1885],
            [-999, -999, -999, ..., 1833, 1851, 1881],
            ...,
            [4951, 4963, 4941, ..., 3358, 3411, 3387],
            [4919, 4932, 4933, ..., 3408, 3481, 3444],
            [4901, 4937, 4937, ..., 3488, 3543, 3470]],
    
           [[1074, 1070, 1070, ...,  764,  740,  739],
            [ 999, 1027, 1019, ...,  729,  732,  759],
            [ 978,  995, 1005, ...,  718,  739,  769],
            ...,
            [ 467,  322,  296, ...,  776,  753,  716],
            [ 376,  289,  278, ...,  641,  640,  615],
            [ 324,  287,  283, ...,  482,  491,  534]]], dtype=int16)

  • nbart_green
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 569 635 682

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1971, 1982, 2003],
            [-999, -999, -999, ..., 1950, 1963, 1993],
            [-999, -999, -999, ..., 1942, 1958, 1985],
            ...,
            [4932, 4943, 4925, ..., 3364, 3423, 3400],
            [4903, 4922, 4912, ..., 3417, 3488, 3455],
            [4890, 4923, 4912, ..., 3505, 3558, 3483]],
    
           [[1635, 1584, 1574, ..., 1074, 1040, 1030],
            [1458, 1479, 1466, ..., 1011, 1015, 1047],
            [1416, 1429, 1445, ..., 1000, 1023, 1047],
            ...,
            [ 486,  335,  295, ..., 1047, 1003,  940],
            [ 405,  292,  275, ...,  833,  811,  790],
            [ 349,  294,  285, ...,  569,  635,  682]]], dtype=int16)

  • nbart_red
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 665 738 845

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2137, 2148, 2168],
            [-999, -999, -999, ..., 2111, 2122, 2158],
            [-999, -999, -999, ..., 2103, 2123, 2150],
            ...,
            [4912, 4938, 4919, ..., 3382, 3447, 3421],
            [4883, 4911, 4901, ..., 3446, 3527, 3483],
            [4880, 4915, 4905, ..., 3538, 3596, 3514]],
    
           [[2268, 2184, 2160, ..., 1366, 1316, 1279],
            [1989, 2010, 2008, ..., 1276, 1268, 1305],
            [1927, 1938, 1961, ..., 1257, 1275, 1313],
            ...,
            [ 486,  328,  294, ..., 1339, 1273, 1198],
            [ 386,  274,  252, ..., 1042, 1016,  959],
            [ 332,  279,  273, ...,  665,  738,  845]]], dtype=int16)

  • nbart_nir
    (time, y, x)
    int16
    -999 -999 -999 ... 2082 2238 2334

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2823, 2837, 2861],
            [-999, -999, -999, ..., 2797, 2813, 2845],
            [-999, -999, -999, ..., 2788, 2806, 2829],
            ...,
            [5169, 5179, 5158, ..., 3927, 3985, 3957],
            [5143, 5160, 5150, ..., 3988, 4065, 4018],
            [5133, 5160, 5150, ..., 4078, 4127, 4049]],
    
           [[3545, 3377, 3295, ..., 2795, 2795, 2833],
            [2969, 2972, 2930, ..., 2687, 2724, 2807],
            [2848, 2847, 2869, ..., 2664, 2713, 2723],
            ...,
            [1514, 1132,  987, ..., 2529, 2356, 2150],
            [1309,  915,  867, ..., 2271, 2186, 2169],
            [1173,  820,  789, ..., 2082, 2238, 2334]]], dtype=int16)

  • nbart_swir_1
    (time, y, x)
    int16
    -999 -999 -999 ... 1830 1883 2178

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1958, 1966, 1988],
            [-999, -999, -999, ..., 1947, 1956, 1983],
            [-999, -999, -999, ..., 1947, 1952, 1978],
            ...,
            [1747, 1758, 1744, ..., 1696, 1724, 1709],
            [1730, 1733, 1716, ..., 1726, 1766, 1743],
            [1702, 1707, 1687, ..., 1780, 1803, 1761]],
    
           [[4288, 4196, 4151, ..., 3679, 3685, 3739],
            [3848, 3908, 3925, ..., 3643, 3688, 3780],
            [3768, 3813, 3860, ..., 3667, 3685, 3769],
            ...,
            [ 971,  695,  588, ..., 3403, 3494, 3418],
            [ 769,  525,  457, ..., 2754, 3002, 2980],
            [ 638,  478,  447, ..., 1830, 1883, 2178]]], dtype=int16)

  • nbart_swir_2
    (time, y, x)
    int16
    -999 -999 -999 ... 990 1184 1331

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1590, 1599, 1617],
            [-999, -999, -999, ..., 1583, 1592, 1618],
            [-999, -999, -999, ..., 1585, 1583, 1610],
            ...,
            [1933, 1938, 1925, ..., 1643, 1677, 1664],
            [1904, 1912, 1901, ..., 1677, 1721, 1696],
            [1878, 1894, 1881, ..., 1735, 1763, 1718]],
    
           [[3109, 3091, 3087, ..., 2571, 2546, 2554],
            [3010, 3036, 3074, ..., 2537, 2549, 2634],
            [2940, 2991, 3045, ..., 2555, 2563, 2686],
            ...,
            [ 541,  351,  305, ..., 2207, 2358, 2308],
            [ 397,  254,  226, ..., 1788, 1917, 1954],
            [ 319,  247,  244, ...,  990, 1184, 1331]]], dtype=int16)

  • nbart_panchromatic
    (time, y, x)
    int16
    -999 -999 -999 -999 ... 583 598 621

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2007, 2012, 2038],
            [-999, -999, -999, ..., 1968, 1988, 2034],
            [-999, -999, -999, ..., 1970, 1982, 2029],
            ...,
            [4886, 4887, 4879, ..., 3382, 3442, 3396],
            [4841, 4846, 4866, ..., 3439, 3529, 3472],
            [4823, 4857, 4853, ..., 3524, 3604, 3520]],
    
           [[1965, 1849, 1795, ..., 1170, 1085, 1095],
            [1595, 1604, 1611, ..., 1049, 1066, 1106],
            [1574, 1555, 1602, ..., 1066, 1068, 1120],
            ...,
            [ 553,  298,  273, ..., 1185, 1093,  995],
            [ 362,  253,  228, ...,  925,  919,  881],
            [ 321,  266,  273, ...,  583,  598,  621]]], dtype=int16)

  • oa_fmask
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    1

    nodata :

    0

    flags_definition :

    {'fmask': {'bits': [0, 1, 2, 3, 4, 5, 6, 7], 'values': {'0': 'nodata', '1': 'valid', '2': 'cloud', '3': 'shadow', '4': 'snow', '5': 'water'}, 'description': 'Fmask'}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            ...,
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0]]], dtype=uint8)

  • oa_nbart_contiguity
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    flags_definition :

    {'contiguous': {'bits': [0], 'values': {'0': False, '1': True}}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_azimuthal_exiting
    (time, y, x)
    float32
    -80.12 -69.12 ... -73.24 -37.26

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -80.11553  ,  -69.122116 ,  -74.337006 , ...,   33.770046 ,
               97.201294 ,  133.90964  ],
            [ -98.09167  ,  -94.05396  ,  -87.45524  , ..., -135.61787  ,
              -76.76864  ,   76.9891   ],
            [-102.43699  , -115.73346  , -107.09284  , ..., -145.66154  ,
             -141.03618  , -102.94009  ],
            ...,
            [  18.39366  ,   13.645186 ,   17.93953  , ...,   60.979168 ,
               83.76587  ,   66.93567  ],
            [  19.606314 ,   11.669298 ,   11.353658 , ...,   88.5983   ,
               61.46648  ,   47.17998  ],
            [  11.797114 ,    2.8835816,    3.9532704, ...,   91.169556 ,
               22.845282 ,   38.14933  ]],
    
           [[  68.512245 ,   70.72293  ,   63.153164 , ...,  -57.384872 ,
              -57.144466 ,   -8.937615 ],
            [  57.491165 ,   53.669292 ,   51.882816 , ...,   62.990677 ,
               32.850204 , -101.98451  ],
            [  41.615105 ,   28.149704 ,   29.797287 , ...,   56.238945 ,
               50.34515  ,  -35.11987  ],
            ...,
            [ -55.327877 ,  -60.8111   ,  -58.213642 , ...,   42.198055 ,
               32.045265 ,   14.97294  ],
            [ -56.430367 ,  -58.39121  ,  -54.94616  , ...,   36.177956 ,
              -21.378311 ,  -20.307953 ],
            [ -52.08491  ,  -50.880047 ,  -45.48003  , ...,  -16.137846 ,
              -73.24443  ,  -37.25841  ]]], dtype=float32)

  • oa_azimuthal_incident
    (time, y, x)
    float32
    -53.29 -43.81 ... 101.3 106.9

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -53.29337 ,  -43.814816,  -49.677975, ...,   73.61261 ,
               83.28998 , -150.34354 ],
            [ -70.001015,  -67.43494 ,  -62.690357, ...,  -89.084045,
              -35.280712,  110.770065],
            [ -77.07651 ,  -90.91911 ,  -84.01256 , ...,  -97.06677 ,
             -102.89715 ,  -62.93791 ],
            ...,
            [  71.86952 ,   65.37238 ,   70.617676, ...,  -37.93936 ,
             -127.73148 ,   55.0821  ],
            [  72.86513 ,   63.776108,   64.33563 , ...,  -86.956635,
               92.59873 ,  138.75403 ],
            [  65.48859 ,   55.617256,   58.376175, ..., -145.5778  ,
               96.301636,  128.84294 ]],
    
           [[ -49.41655 ,  -40.01738 ,  -45.558876, ...,   77.89437 ,
               87.38188 , -146.33037 ],
            [ -65.91871 ,  -64.03974 ,  -59.186226, ...,  -84.63618 ,
              -31.004824,  115.21529 ],
            [ -72.751   ,  -87.00204 ,  -80.26874 , ...,  -92.55007 ,
              -98.49117 ,  -58.568077],
            ...,
            [  76.22296 ,   69.71172 ,   75.02115 , ...,  -32.531303,
             -121.95462 ,   60.88784 ],
            [  77.265816,   68.14368 ,   68.7059  , ...,  -81.56631 ,
               98.153   ,  143.99895 ],
            [  69.8667  ,   59.935596,   62.67588 , ..., -138.76768 ,
              101.29737 ,  106.94179 ]]], dtype=float32)

  • oa_combined_terrain_shadow
    (time, y, x)
    uint8
    1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_exiting_angle
    (time, y, x)
    float32
    9.029 9.148 9.09 ... 8.081 10.43

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 9.028952 ,  9.148497 ,  9.090177 , ...,  3.2647538,
              3.62325  ,  4.391917 ],
            [ 8.894047 ,  8.901706 ,  8.9225235, ...,  3.2806778,
              3.5062037,  4.4214954],
            [ 8.756343 ,  8.535571 ,  8.580602 , ...,  3.3433805,
              3.560637 ,  4.636894 ],
            ...,
            [11.375915 , 11.375726 , 11.181578 , ...,  5.0770392,
              4.6290197,  6.9061594],
            [11.1641655, 11.834425 , 12.261691 , ...,  3.2524686,
              5.314759 ,  7.5270324],
            [12.537902 , 13.934173 , 14.685269 , ...,  3.791945 ,
              6.8892117,  7.5075154]],
    
           [[ 2.4255953,  2.4921818,  2.615967 , ...,  7.997434 ,
              7.613728 ,  6.8861303],
            [ 2.4387937,  2.5729144,  2.7521896, ...,  8.068149 ,
              7.777022 ,  6.803534 ],
            [ 2.8313277,  3.079027 ,  3.2421515, ...,  8.0514555,
              7.821932 ,  6.652463 ],
            ...,
            [ 5.45706  ,  5.047012 ,  5.174448 , ..., 15.872423 ,
             13.216334 , 13.8058195],
            [ 5.292245 ,  5.412933 ,  5.899394 , ..., 12.911547 ,
             10.252516 , 11.655185 ],
            [ 6.263888 ,  7.0900445,  8.104066 , ...,  9.325176 ,
              8.080662 , 10.431503 ]]], dtype=float32)

  • oa_incident_angle
    (time, y, x)
    float32
    40.0 40.23 40.24 ... 37.89 35.61

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.998978, 40.23141 , 40.237175, ..., 37.26962 , 37.826603,
             38.7017  ],
            [39.77791 , 39.912277, 40.08276 , ..., 37.007404, 37.392212,
             38.432606],
            [39.88898 , 39.73803 , 39.905514, ..., 36.956375, 37.170578,
             38.289707],
            ...,
            [39.404842, 39.735092, 39.47829 , ..., 31.522882, 33.095287,
             32.230022],
            [39.361443, 39.853664, 39.843723, ..., 33.740055, 35.684074,
             34.42446 ],
            [39.81281 , 40.565845, 40.41699 , ..., 36.779434, 37.866028,
             35.635822]],
    
           [[40.41124 , 40.656933, 40.68124 , ..., 37.69816 , 38.263546,
             39.134884],
            [40.19523 , 40.337387, 40.529827, ..., 37.413853, 37.801704,
             38.827084],
            [40.35013 , 40.206455, 40.390034, ..., 37.354183, 37.558178,
             38.643627],
            ...,
            [39.349937, 39.697414, 39.442852, ..., 32.172   , 33.430336,
             32.340042],
            [39.322533, 39.776737, 39.72588 , ..., 34.17244 , 35.808483,
             34.376877],
            [39.66718 , 40.32737 , 40.102894, ..., 37.021255, 37.88646 ,
             35.605076]]], dtype=float32)

  • oa_relative_azimuth
    (time, y, x)
    float32
    32.23 32.23 32.23 ... 217.7 217.7

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 32.232582,  32.23285 ,  32.233112, ...,  32.693703,
              32.69397 ,  32.69425 ],
            [ 32.23295 ,  32.23321 ,  32.233475, ...,  32.69407 ,
              32.69434 ,  32.694626],
            [ 32.233315,  32.233578,  32.23384 , ...,  32.694443,
              32.694706,  32.694992],
            ...,
            [ 32.975964,  32.976234,  32.976505, ...,  33.447357,
              33.447643,  33.447918],
            [ 32.976337,  32.976612,  32.976875, ...,  33.447735,
              33.448017,  33.44829 ],
            [ 32.976727,  32.976997,  32.977264, ...,  33.44813 ,
              33.448418,  33.44869 ]],
    
           [[216.49684 , 216.49713 , 216.49739 , ..., 216.99608 ,
             216.99635 , 216.99666 ],
            [216.4972  , 216.49747 , 216.49776 , ..., 216.99643 ,
             216.99673 , 216.99702 ],
            [216.49756 , 216.49783 , 216.49811 , ..., 216.9968  ,
             216.99709 , 216.99738 ],
            ...,
            [217.21596 , 217.21623 , 217.21652 , ..., 217.72452 ,
             217.72482 , 217.72513 ],
            [217.21632 , 217.21661 , 217.21689 , ..., 217.72487 ,
             217.72519 , 217.7255  ],
            [217.2167  , 217.217   , 217.21727 , ..., 217.72527 ,
             217.72557 , 217.72588 ]]], dtype=float32)

  • oa_relative_slope
    (time, y, x)
    float32
    26.82 25.31 24.66 ... 141.0 171.0

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[  26.822166,   25.307304,   24.659027, ...,   39.842567,
               32.689396,   26.666718],
            [  28.09065 ,   26.619028,   24.764885, ...,   46.53382 ,
               41.487926,   33.780968],
            [  25.360477,   24.81435 ,   23.08028 , ...,   48.594772,
               47.248375,   40.002182],
            ...,
            [  53.47586 ,   51.727196,   52.678143, ..., -169.05045 ,
               43.374695,  110.16484 ],
            [  53.258812,   52.106808,   52.98198 , ...,  116.92698 ,
               94.31888 ,  100.24516 ],
            [  53.69147 ,   52.733677,   54.422905, ...,   82.8719  ,
               73.45635 ,   90.69361 ]],
    
           [[-117.928795, -110.74031 , -108.71204 , ..., -144.39311 ,
             -141.10318 , -137.39276 ],
            [-123.409874, -117.70903 , -111.069046, ..., -147.62685 ,
             -145.64392 , -142.8002  ],
            [-114.366104, -115.15174 , -110.066025, ..., -148.78902 ,
             -148.83632 , -148.09137 ],
            ...,
            [ 131.55084 ,  130.52283 ,  133.23479 , ..., -144.8613  ,
             -160.54012 ,   45.9149  ],
            [ 133.69618 ,  126.53488 ,  123.652054, ..., -152.83937 ,
             -173.85126 ,  135.26845 ],
            [ 121.951614,  110.81564 ,  108.15591 , ..., -122.62984 ,
              140.97137 ,  170.97281 ]]], dtype=float32)

  • oa_satellite_azimuth
    (time, y, x)
    float32
    100.1 100.1 100.1 ... 280.1 280.1

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[100.1332  , 100.13319 , 100.13318 , ..., 100.11447 ,
             100.114456, 100.11445 ],
            [100.13323 , 100.13322 , 100.13321 , ..., 100.1145  ,
             100.114494, 100.11449 ],
            [100.13326 , 100.13325 , 100.13324 , ..., 100.11453 ,
             100.114525, 100.11452 ],
            ...,
            [100.19855 , 100.19853 , 100.198524, ..., 100.179245,
             100.17924 , 100.17922 ],
            [100.19858 , 100.19857 , 100.198555, ..., 100.17928 ,
             100.179276, 100.17926 ],
            [100.198616, 100.1986  , 100.198586, ..., 100.17931 ,
             100.1793  , 100.17929 ]],
    
           [[280.09695 , 280.09692 , 280.09692 , ..., 280.07846 ,
             280.07843 , 280.07843 ],
            [280.097   , 280.09695 , 280.09695 , ..., 280.0785  ,
             280.0785  , 280.07846 ],
            [280.09702 , 280.09702 , 280.097   , ..., 280.07852 ,
             280.07852 , 280.0785  ],
            ...,
            [280.16147 , 280.16144 , 280.1614  , ..., 280.1424  ,
             280.1424  , 280.1424  ],
            [280.1615  , 280.16147 , 280.16144 , ..., 280.14243 ,
             280.14243 , 280.14243 ],
            [280.16153 , 280.1615  , 280.16147 , ..., 280.14246 ,
             280.14246 , 280.14246 ]]], dtype=float32)

  • oa_satellite_view
    (time, y, x)
    float32
    9.022 9.02 9.017 ... 7.737 7.74

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[9.02193  , 9.019681 , 9.017433 , ..., 5.130571 , 5.1282773,
             5.125959 ],
            [9.021114 , 9.018867 , 9.016618 , ..., 5.1297407, 5.1274476,
             5.1251235],
            [9.0203   , 9.018051 , 9.015802 , ..., 5.128911 , 5.1266174,
             5.1242886],
            ...,
            [7.31853  , 7.3162613, 7.3139925, ..., 3.4020782, 3.3997495,
             3.3974464],
            [7.31763  , 7.3153787, 7.3131275, ..., 3.401222 , 3.3989155,
             3.3966124],
            [7.316651 , 7.314382 , 7.3121133, ..., 3.400168 , 3.397848 ,
             3.3955448]],
    
           [[2.1344194, 2.1367428, 2.1390662, ..., 6.0918517, 6.094148 ,
             6.096469 ],
            [2.1352227, 2.137546 , 2.1398695, ..., 6.092644 , 6.094941 ,
             6.097266 ],
            [2.1360261, 2.1383495, 2.140673 , ..., 6.0934367, 6.095733 ,
             6.0980635],
            ...,
            [3.803363 , 3.805676 , 3.8079891, ..., 7.733246 , 7.7355447,
             7.7378206],
            [3.804241 , 3.806536 , 3.808831 , ..., 7.734052 , 7.7363305,
             7.738606 ],
            [3.8051991, 3.8075123, 3.8098252, ..., 7.7350545, 7.737346 ,
             7.7396216]]], dtype=float32)

  • oa_solar_azimuth
    (time, y, x)
    float32
    67.9 67.9 67.9 ... 62.42 62.42

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[67.90062 , 67.900345, 67.90006 , ..., 67.42077 , 67.42049 ,
             67.4202  ],
            [67.90028 , 67.90001 , 67.899734, ..., 67.42043 , 67.42015 ,
             67.41986 ],
            [67.89995 , 67.89967 , 67.8994  , ..., 67.4201  , 67.419815,
             67.41952 ],
            ...,
            [67.22258 , 67.2223  , 67.222015, ..., 66.73189 , 66.73159 ,
             66.73131 ],
            [67.222244, 67.221954, 67.22168 , ..., 66.73155 , 66.731255,
             66.73097 ],
            [67.221886, 67.2216  , 67.22132 , ..., 66.73118 , 66.73089 ,
             66.7306  ]],
    
           [[63.60011 , 63.59981 , 63.599514, ..., 63.082386, 63.082085,
             63.08177 ],
            [63.59979 , 63.59949 , 63.599194, ..., 63.082058, 63.081757,
             63.081444],
            [63.599464, 63.599167, 63.59887 , ..., 63.08173 , 63.08143 ,
             63.081112],
            ...,
            [62.94549 , 62.94519 , 62.944885, ..., 62.41788 , 62.41756 ,
             62.417255],
            [62.945164, 62.94486 , 62.94456 , ..., 62.417553, 62.417236,
             62.41693 ],
            [62.94482 , 62.944515, 62.944214, ..., 62.417194, 62.416878,
             62.416573]]], dtype=float32)

  • oa_solar_zenith
    (time, y, x)
    float32
    39.49 39.49 39.49 ... 39.6 39.6

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.487972, 39.48771 , 39.48745 , ..., 39.04551 , 39.04525 ,
             39.044994],
            [39.48805 , 39.487785, 39.487526, ..., 39.045586, 39.045326,
             39.04507 ],
            [39.48812 , 39.48786 , 39.4876  , ..., 39.045662, 39.045403,
             39.045147],
            ...,
            [39.63633 , 39.63607 , 39.63581 , ..., 39.195976, 39.195724,
             39.19546 ],
            [39.636402, 39.636143, 39.635883, ..., 39.196053, 39.1958  ,
             39.19554 ],
            [39.636463, 39.636204, 39.635944, ..., 39.196114, 39.195854,
             39.195595]],
    
           [[39.84592 , 39.845665, 39.845406, ..., 39.414013, 39.413757,
             39.413506],
            [39.846012, 39.845757, 39.8455  , ..., 39.41411 , 39.413853,
             39.413605],
            [39.846107, 39.84585 , 39.845596, ..., 39.414204, 39.413948,
             39.4137  ],
            ...,
            [40.032944, 40.032692, 40.032436, ..., 39.60342 , 39.603172,
             39.602917],
            [40.033035, 40.032784, 40.03253 , ..., 39.603516, 39.60327 ,
             39.603016],
            [40.033115, 40.032864, 40.032608, ..., 39.603596, 39.603344,
             39.603092]]], dtype=float32)

  • oa_time_delta
    (time, y, x)
    float32
    -0.4956 -0.4971 ... 1.101 1.099

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-0.49559563, -0.4970854 , -0.49857512, ..., -2.9306152 ,
             -2.9321241 , -2.9334474 ],
            [-0.49143195, -0.49292168, -0.49441144, ..., -2.9264584 ,
             -2.9279675 , -2.929252  ],
            [-0.48726824, -0.48875797, -0.49024773, ..., -2.9223018 ,
             -2.923811  , -2.9250565 ],
            ...,
            [ 7.86643   ,  7.864933  ,  7.863436  , ...,  5.4181333 ,
              5.416802  ,  5.4152856 ],
            [ 7.870569  ,  7.869079  ,  7.867589  , ...,  5.422319  ,
              5.420964  ,  5.4194474 ],
            [ 7.874677  ,  7.8731804 ,  7.8716836 , ...,  5.4264264 ,
              5.4250336 ,  5.423517  ]],
    
           [[-4.8207088 , -4.8221984 , -4.823688  , ..., -7.255672  ,
             -7.2571807 , -7.2585034 ],
            [-4.816544  , -4.8180337 , -4.8195233 , ..., -7.251514  ,
             -7.2530227 , -7.254307  ],
            [-4.812379  , -4.8138685 , -4.815358  , ..., -7.247356  ,
             -7.248864  , -7.2501097 ],
            ...,
            [ 3.5420945 ,  3.5405984 ,  3.539102  , ...,  1.0940956 ,
              1.0927645 ,  1.0912488 ],
            [ 3.5462334 ,  3.5447443 ,  3.543255  , ...,  1.0982829 ,
              1.0969281 ,  1.0954124 ],
            [ 3.5503414 ,  3.5488453 ,  3.547349  , ...,  1.1023915 ,
              1.1009991 ,  1.0994833 ]]], dtype=float32)

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449'], dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (2)

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

The impact of dask on ODC

From the above we can see that specifying dask_chunks in datacube.load() splits up the load() operation into a set of chunk shaped arrays and delayed tasks. Dask can now perform those tasks in parallel. Dask will only compute the results for those parts of the data we are using but we can force the computation of all the delayed tasks using compute().

There is a lot more opportunity than described in this simple example but let's just focus on the impact of dask on ODC for this simple case.

The time period and ROI are far too small to be interesting so let's change our time range to a few months of data.

set_time = (set_time[0], parse(set_time[0]) + relativedelta(months=6))
# set_time = ("2021-01-01", "2021-06-30")
set_time

('2020-02-01', datetime.datetime(2020, 8, 1, 0, 0))

We skip loading this longer time range (larger data selection) without dask because it can take many minutes and may use more than the available memory in the Jupyter node.

Let's enable dask and then do the load. We're chunking by time (length one) so dask will be able to load each time slice in parallel. The data variables are also independent so will be done in parallel as well.

if client is None:
    cluster = LocalCluster()
    client = Client(cluster)
    configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);
    display(notebook_utils.localcluster_dashboard(client=client, server=easi.hub))
else:
    client.restart()

'https://hub.csiro.easi-eo.solutions/user/csiro-csiro-aad_pag064@csiro.au/proxy/8787/status'

%%time

chunks = {"time":1}

dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, ###### THIS IS THE ONLY LINE CHANGED. #####
            group_by=group_by,
        )
dataset

CPU times: user 287 ms, sys: 26.3 ms, total: 313 ms
Wall time: 330 ms

<xarray.Dataset>
Dimensions:                     (time: 22, y: 2013, x: 1709)
Coordinates:
  * time                        (time) datetime64[ns] 2020-02-08T23:56:55.714...
  * y                           (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06
  * x                           (x) float64 1.328e+06 1.328e+06 ... 1.379e+06
    spatial_ref                 int32 3577
Data variables: (12/22)
    nbart_coastal_aerosol       (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_blue                  (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_green                 (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_red                   (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_nir                   (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    nbart_swir_1                (time, y, x) int16 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    ...                          ...
    oa_relative_slope           (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_satellite_azimuth        (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_satellite_view           (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_solar_azimuth            (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_solar_zenith             (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
    oa_time_delta               (time, y, x) float32 dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

• Dimensions:
  • time: 22
  • y: 2013
  • x: 1709
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 ... 2...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000',
           '2020-02-24T23:56:51.968040000', '2020-03-03T00:03:00.297499000',
           '2020-03-11T23:56:45.657477000', '2020-03-19T00:02:52.920718000',
           '2020-03-27T23:56:36.905801000', '2020-04-04T00:02:43.534168000',
           '2020-04-12T23:56:29.492153000', '2020-04-20T00:02:37.218009000',
           '2020-04-28T23:56:21.738632000', '2020-05-06T00:02:28.332833000',
           '2020-05-14T23:56:17.682743000', '2020-05-22T00:02:29.991429000',
           '2020-05-30T23:56:21.787385000', '2020-06-07T00:02:37.301210000',
           '2020-06-15T23:56:31.825059000', '2020-06-23T00:02:46.402339000',
           '2020-07-01T23:56:39.787669000', '2020-07-09T00:02:53.484642000',
           '2020-07-17T23:56:45.798382000', '2020-07-25T00:02:58.653767000'],
          dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Data variables: (22)
  • nbart_coastal_aerosol
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_blue
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_green
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_red
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_nir
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_swir_1
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_swir_2
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • nbart_panchromatic
    (time, y, x)
    int16
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 144.36 MiB 6.56 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type int16 numpy.ndarray

  • oa_fmask
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    0

    flags_definition :

    {'fmask': {'bits': [0, 1, 2, 3, 4, 5, 6, 7], 'values': {'0': 'nodata', '1': 'valid', '2': 'cloud', '3': 'shadow', '4': 'snow', '5': 'water'}, 'description': 'Fmask'}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 72.18 MiB 3.28 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type uint8 numpy.ndarray

  • oa_nbart_contiguity
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    255

    flags_definition :

    {'contiguous': {'bits': [0], 'values': {'0': False, '1': True}}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 72.18 MiB 3.28 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type uint8 numpy.ndarray

  • oa_azimuthal_exiting
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_azimuthal_incident
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_combined_terrain_shadow
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    255

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 72.18 MiB 3.28 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type uint8 numpy.ndarray

  • oa_exiting_angle
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_incident_angle
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_relative_azimuth
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_relative_slope
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_satellite_azimuth
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_satellite_view
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_solar_azimuth
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_solar_zenith
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

  • oa_time_delta
    (time, y, x)
    float32
    dask.array<chunksize=(1, 2013, 1709), meta=np.ndarray>

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 288.71 MiB 13.12 MiB Shape (22, 2013, 1709) (1, 2013, 1709) Dask graph 22 chunks in 1 graph layer Data type float32 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449',
                   '2020-02-24 23:56:51.968040', '2020-03-03 00:03:00.297499',
                   '2020-03-11 23:56:45.657477', '2020-03-19 00:02:52.920718',
                   '2020-03-27 23:56:36.905801', '2020-04-04 00:02:43.534168',
                   '2020-04-12 23:56:29.492153', '2020-04-20 00:02:37.218009',
                   '2020-04-28 23:56:21.738632', '2020-05-06 00:02:28.332833',
                   '2020-05-14 23:56:17.682743', '2020-05-22 00:02:29.991429',
                   '2020-05-30 23:56:21.787385', '2020-06-07 00:02:37.301210',
                   '2020-06-15 23:56:31.825059', '2020-06-23 00:02:46.402339',
                   '2020-07-01 23:56:39.787669', '2020-07-09 00:02:53.484642',
                   '2020-07-17 23:56:45.798382', '2020-07-25 00:02:58.653767'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (2)

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

Woah!! that was fast - but we didn't actually compute anything so no load has occurred and all tasks are pending. Open up the Data Variables, click the stacked cylinders and take a look at the delayed task counts. These exist for every variable.

Let's visualise the task graph for the red band.

dataset[alias['red']].data.visualize()

Well that's not as useful, is it!

You should just be able to make out that each of the chunks are able to independently load(). time chunk is length 1 so these are individual times. This holds true for all the bands so dask can spread these out across multiple threads.

Tip: Visualising task graphs is less effective as your task graph complexity increases. You may need to use simpler examples to see what is going on.

Let's get the actual data

%%time
actual_dataset = dataset.compute()
actual_dataset

/env/lib/python3.10/site-packages/rasterio/warp.py:344: NotGeoreferencedWarning: Dataset has no geotransform, gcps, or rpcs. The identity matrix will be returned.
  _reproject(

CPU times: user 14.5 s, sys: 5.37 s, total: 19.9 s
Wall time: 1min 38s

<xarray.Dataset>
Dimensions:                     (time: 22, y: 2013, x: 1709)
Coordinates:
  * time                        (time) datetime64[ns] 2020-02-08T23:56:55.714...
  * y                           (y) float64 -3.988e+06 -3.988e+06 ... -4.049e+06
  * x                           (x) float64 1.328e+06 1.328e+06 ... 1.379e+06
    spatial_ref                 int32 3577
Data variables: (12/22)
    nbart_coastal_aerosol       (time, y, x) int16 -999 -999 -999 ... -999 -999
    nbart_blue                  (time, y, x) int16 -999 -999 -999 ... -999 -999
    nbart_green                 (time, y, x) int16 -999 -999 -999 ... -999 -999
    nbart_red                   (time, y, x) int16 -999 -999 -999 ... -999 -999
    nbart_nir                   (time, y, x) int16 -999 -999 -999 ... -999 -999
    nbart_swir_1                (time, y, x) int16 -999 -999 -999 ... -999 -999
    ...                          ...
    oa_relative_slope           (time, y, x) float32 26.82 25.31 ... -155.9
    oa_satellite_azimuth        (time, y, x) float32 100.1 100.1 ... 280.2 280.2
    oa_satellite_view           (time, y, x) float32 9.022 9.02 ... 7.95 7.952
    oa_solar_azimuth            (time, y, x) float32 67.9 67.9 ... 35.28 35.28
    oa_solar_zenith             (time, y, x) float32 39.49 39.49 ... 64.21 64.21
    oa_time_delta               (time, y, x) float32 -0.4956 -0.4971 ... 1.008
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

• Dimensions:
  • time: 22
  • y: 2013
  • x: 1709
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2020-02-08T23:56:55.714845 ... 2...

    units :

    seconds since 1970-01-01 00:00:00

    array(['2020-02-08T23:56:55.714845000', '2020-02-16T00:03:05.179449000',
           '2020-02-24T23:56:51.968040000', '2020-03-03T00:03:00.297499000',
           '2020-03-11T23:56:45.657477000', '2020-03-19T00:02:52.920718000',
           '2020-03-27T23:56:36.905801000', '2020-04-04T00:02:43.534168000',
           '2020-04-12T23:56:29.492153000', '2020-04-20T00:02:37.218009000',
           '2020-04-28T23:56:21.738632000', '2020-05-06T00:02:28.332833000',
           '2020-05-14T23:56:17.682743000', '2020-05-22T00:02:29.991429000',
           '2020-05-30T23:56:21.787385000', '2020-06-07T00:02:37.301210000',
           '2020-06-15T23:56:31.825059000', '2020-06-23T00:02:46.402339000',
           '2020-07-01T23:56:39.787669000', '2020-07-09T00:02:53.484642000',
           '2020-07-17T23:56:45.798382000', '2020-07-25T00:02:58.653767000'],
          dtype='datetime64[ns]')

  • y
    (y)
    float64
    -3.988e+06 ... -4.049e+06

    units :

    metre

    resolution :

    -30.0

    crs :

    EPSG:3577

    array([-3988395., -3988425., -3988455., ..., -4048695., -4048725., -4048755.])

  • x
    (x)
    float64
    1.328e+06 1.328e+06 ... 1.379e+06

    units :

    metre

    resolution :

    30.0

    crs :

    EPSG:3577

    array([1327605., 1327635., 1327665., ..., 1378785., 1378815., 1378845.])

  • spatial_ref
    ()
    int32
    3577

    spatial_ref :

    PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

    grid_mapping_name :

    albers_conical_equal_area

    array(3577, dtype=int32)

• Data variables: (22)
  • nbart_coastal_aerosol
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1824, 1832, 1861],
            [-999, -999, -999, ..., 1812, 1826, 1858],
            [-999, -999, -999, ..., 1801, 1819, 1846],
            ...,
            [4977, 4976, 4969, ..., 3373, 3424, 3403],
            [4946, 4947, 4948, ..., 3421, 3497, 3461],
            [4918, 4947, 4946, ..., 3507, 3553, 3487]],
    
           [[ 868,  860,  862, ...,  644,  627,  630],
            [ 827,  857,  857, ...,  615,  618,  648],
            [ 815,  831,  838, ...,  608,  624,  651],
            ...,
            [ 468,  350,  327, ...,  642,  644,  623],
            [ 394,  328,  310, ...,  569,  561,  545],
            [ 353,  327,  311, ...,  440,  478,  499]],
    
           [[-999, -999, -999, ...,  736,  731,  733],
            [-999, -999, -999, ...,  708,  709,  740],
            [-999, -999, -999, ...,  698,  717,  746],
            ...,
    ...
            ...,
            [ 199,  201,  193, ..., -999, -999, -999],
            [ 186,  196,  181, ..., -999, -999, -999],
            [ 170,  194,  182, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ...,  272,  252,  232],
            [-999, -999, -999, ...,  229,  208,  204],
            [-999, -999, -999, ...,  201,  181,  185],
            ...,
            [1165, 1333, 1274, ...,  206,  205,  217],
            [ 952, 1226, 1166, ...,  223,  235,  229],
            [ 759, 1089, 1105, ...,  177,  194,  200]],
    
           [[ 315,  325,  332, ...,  194,  165,  158],
            [ 322,  332,  334, ...,  169,  154,  157],
            [ 335,  346,  355, ...,  157,  146,  157],
            ...,
            [ 636,  597,  582, ..., -999, -999, -999],
            [ 659,  657,  665, ..., -999, -999, -999],
            [ 659,  697,  724, ..., -999, -999, -999]]], dtype=int16)

  • nbart_blue
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1854, 1868, 1889],
            [-999, -999, -999, ..., 1839, 1858, 1885],
            [-999, -999, -999, ..., 1833, 1851, 1881],
            ...,
            [4951, 4963, 4941, ..., 3358, 3411, 3387],
            [4919, 4932, 4933, ..., 3408, 3481, 3444],
            [4901, 4937, 4937, ..., 3488, 3543, 3470]],
    
           [[1074, 1070, 1070, ...,  764,  740,  739],
            [ 999, 1027, 1019, ...,  729,  732,  759],
            [ 978,  995, 1005, ...,  718,  739,  769],
            ...,
            [ 467,  322,  296, ...,  776,  753,  716],
            [ 376,  289,  278, ...,  641,  640,  615],
            [ 324,  287,  283, ...,  482,  491,  534]],
    
           [[-999, -999, -999, ...,  868,  856,  856],
            [-999, -999, -999, ...,  831,  840,  872],
            [-999, -999, -999, ...,  823,  846,  867],
            ...,
    ...
            ...,
            [ 213,  217,  203, ..., -999, -999, -999],
            [ 203,  208,  185, ..., -999, -999, -999],
            [ 184,  210,  193, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ...,  317,  289,  278],
            [-999, -999, -999, ...,  281,  264,  270],
            [-999, -999, -999, ...,  263,  248,  256],
            ...,
            [1209, 1384, 1314, ...,  246,  269,  263],
            [ 978, 1290, 1243, ...,  278,  290,  286],
            [ 776, 1129, 1149, ...,  177,  235,  242]],
    
           [[ 346,  354,  355, ...,  238,  192,  192],
            [ 345,  355,  360, ...,  199,  186,  193],
            [ 359,  365,  374, ...,  186,  182,  193],
            ...,
            [ 670,  640,  621, ..., -999, -999, -999],
            [ 684,  691,  697, ..., -999, -999, -999],
            [ 678,  714,  731, ..., -999, -999, -999]]], dtype=int16)

  • nbart_green
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1971, 1982, 2003],
            [-999, -999, -999, ..., 1950, 1963, 1993],
            [-999, -999, -999, ..., 1942, 1958, 1985],
            ...,
            [4932, 4943, 4925, ..., 3364, 3423, 3400],
            [4903, 4922, 4912, ..., 3417, 3488, 3455],
            [4890, 4923, 4912, ..., 3505, 3558, 3483]],
    
           [[1635, 1584, 1574, ..., 1074, 1040, 1030],
            [1458, 1479, 1466, ..., 1011, 1015, 1047],
            [1416, 1429, 1445, ..., 1000, 1023, 1047],
            ...,
            [ 486,  335,  295, ..., 1047, 1003,  940],
            [ 405,  292,  275, ...,  833,  811,  790],
            [ 349,  294,  285, ...,  569,  635,  682]],
    
           [[-999, -999, -999, ..., 1187, 1151, 1148],
            [-999, -999, -999, ..., 1111, 1122, 1156],
            [-999, -999, -999, ..., 1104, 1134, 1152],
            ...,
    ...
            ...,
            [ 306,  298,  285, ..., -999, -999, -999],
            [ 286,  294,  262, ..., -999, -999, -999],
            [ 271,  304,  278, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ...,  639,  607,  629],
            [-999, -999, -999, ...,  602,  590,  600],
            [-999, -999, -999, ...,  576,  560,  579],
            ...,
            [1286, 1481, 1406, ...,  557,  607,  603],
            [1050, 1365, 1300, ...,  487,  547,  566],
            [ 840, 1200, 1218, ...,  302,  374,  391]],
    
           [[ 530,  541,  536, ...,  572,  513,  518],
            [ 522,  532,  533, ...,  521,  494,  496],
            [ 526,  533,  541, ...,  493,  479,  494],
            ...,
            [ 729,  687,  666, ..., -999, -999, -999],
            [ 741,  711,  709, ..., -999, -999, -999],
            [ 774,  804,  799, ..., -999, -999, -999]]], dtype=int16)

  • nbart_red
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2137, 2148, 2168],
            [-999, -999, -999, ..., 2111, 2122, 2158],
            [-999, -999, -999, ..., 2103, 2123, 2150],
            ...,
            [4912, 4938, 4919, ..., 3382, 3447, 3421],
            [4883, 4911, 4901, ..., 3446, 3527, 3483],
            [4880, 4915, 4905, ..., 3538, 3596, 3514]],
    
           [[2268, 2184, 2160, ..., 1366, 1316, 1279],
            [1989, 2010, 2008, ..., 1276, 1268, 1305],
            [1927, 1938, 1961, ..., 1257, 1275, 1313],
            ...,
            [ 486,  328,  294, ..., 1339, 1273, 1198],
            [ 386,  274,  252, ..., 1042, 1016,  959],
            [ 332,  279,  273, ...,  665,  738,  845]],
    
           [[-999, -999, -999, ..., 1561, 1509, 1483],
            [-999, -999, -999, ..., 1464, 1466, 1483],
            [-999, -999, -999, ..., 1444, 1453, 1459],
            ...,
    ...
            ...,
            [ 312,  313,  289, ..., -999, -999, -999],
            [ 296,  303,  269, ..., -999, -999, -999],
            [ 274,  315,  290, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ...,  443,  363,  364],
            [-999, -999, -999, ...,  368,  349,  355],
            [-999, -999, -999, ...,  339,  334,  344],
            ...,
            [1312, 1505, 1418, ...,  388,  390,  384],
            [1052, 1409, 1342, ...,  418,  476,  440],
            [ 838, 1224, 1226, ...,  269,  304,  416]],
    
           [[ 390,  398,  390, ...,  358,  234,  232],
            [ 390,  395,  392, ...,  257,  223,  236],
            [ 394,  400,  402, ...,  224,  226,  236],
            ...,
            [ 753,  725,  706, ..., -999, -999, -999],
            [ 768,  752,  743, ..., -999, -999, -999],
            [ 804,  841,  839, ..., -999, -999, -999]]], dtype=int16)

  • nbart_nir
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2823, 2837, 2861],
            [-999, -999, -999, ..., 2797, 2813, 2845],
            [-999, -999, -999, ..., 2788, 2806, 2829],
            ...,
            [5169, 5179, 5158, ..., 3927, 3985, 3957],
            [5143, 5160, 5150, ..., 3988, 4065, 4018],
            [5133, 5160, 5150, ..., 4078, 4127, 4049]],
    
           [[3545, 3377, 3295, ..., 2795, 2795, 2833],
            [2969, 2972, 2930, ..., 2687, 2724, 2807],
            [2848, 2847, 2869, ..., 2664, 2713, 2723],
            ...,
            [1514, 1132,  987, ..., 2529, 2356, 2150],
            [1309,  915,  867, ..., 2271, 2186, 2169],
            [1173,  820,  789, ..., 2082, 2238, 2334]],
    
           [[-999, -999, -999, ..., 2778, 2764, 2792],
            [-999, -999, -999, ..., 2688, 2729, 2803],
            [-999, -999, -999, ..., 2694, 2755, 2786],
            ...,
    ...
            ...,
            [1914, 1818, 1683, ..., -999, -999, -999],
            [1861, 1765, 1565, ..., -999, -999, -999],
            [1582, 1634, 1426, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ..., 4473, 4711, 4760],
            [-999, -999, -999, ..., 4610, 4606, 4612],
            [-999, -999, -999, ..., 4599, 4450, 4496],
            ...,
            [2280, 2488, 2374, ..., 3466, 4120, 4402],
            [2028, 2340, 2274, ..., 3284, 3849, 3981],
            [1762, 2136, 2135, ..., 1809, 2174, 2290]],
    
           [[3038, 3032, 2972, ..., 5440, 5777, 5783],
            [2759, 2817, 2828, ..., 5723, 5660, 5524],
            [2766, 2798, 2841, ..., 5694, 5426, 5395],
            ...,
            [2433, 2369, 2178, ..., -999, -999, -999],
            [2467, 2354, 2240, ..., -999, -999, -999],
            [2377, 2296, 2204, ..., -999, -999, -999]]], dtype=int16)

  • nbart_swir_1
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1958, 1966, 1988],
            [-999, -999, -999, ..., 1947, 1956, 1983],
            [-999, -999, -999, ..., 1947, 1952, 1978],
            ...,
            [1747, 1758, 1744, ..., 1696, 1724, 1709],
            [1730, 1733, 1716, ..., 1726, 1766, 1743],
            [1702, 1707, 1687, ..., 1780, 1803, 1761]],
    
           [[4288, 4196, 4151, ..., 3679, 3685, 3739],
            [3848, 3908, 3925, ..., 3643, 3688, 3780],
            [3768, 3813, 3860, ..., 3667, 3685, 3769],
            ...,
            [ 971,  695,  588, ..., 3403, 3494, 3418],
            [ 769,  525,  457, ..., 2754, 3002, 2980],
            [ 638,  478,  447, ..., 1830, 1883, 2178]],
    
           [[-999, -999, -999, ..., 3812, 3801, 3867],
            [-999, -999, -999, ..., 3739, 3785, 3867],
            [-999, -999, -999, ..., 3749, 3766, 3820],
            ...,
    ...
            ...,
            [ 868,  866,  802, ..., -999, -999, -999],
            [ 774,  800,  719, ..., -999, -999, -999],
            [ 672,  800,  764, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ..., 1557, 1512, 1541],
            [-999, -999, -999, ..., 1498, 1473, 1473],
            [-999, -999, -999, ..., 1463, 1432, 1454],
            ...,
            [2002, 2238, 2098, ..., 1708, 1844, 1809],
            [1650, 2071, 1999, ..., 1523, 1782, 1715],
            [1344, 1813, 1834, ...,  889, 1130, 1107]],
    
           [[1399, 1431, 1404, ..., 1926, 1865, 1858],
            [1353, 1383, 1372, ..., 1880, 1791, 1731],
            [1344, 1365, 1372, ..., 1815, 1714, 1690],
            ...,
            [1367, 1379, 1320, ..., -999, -999, -999],
            [1324, 1336, 1283, ..., -999, -999, -999],
            [1313, 1387, 1354, ..., -999, -999, -999]]], dtype=int16)

  • nbart_swir_2
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 1590, 1599, 1617],
            [-999, -999, -999, ..., 1583, 1592, 1618],
            [-999, -999, -999, ..., 1585, 1583, 1610],
            ...,
            [1933, 1938, 1925, ..., 1643, 1677, 1664],
            [1904, 1912, 1901, ..., 1677, 1721, 1696],
            [1878, 1894, 1881, ..., 1735, 1763, 1718]],
    
           [[3109, 3091, 3087, ..., 2571, 2546, 2554],
            [3010, 3036, 3074, ..., 2537, 2549, 2634],
            [2940, 2991, 3045, ..., 2555, 2563, 2686],
            ...,
            [ 541,  351,  305, ..., 2207, 2358, 2308],
            [ 397,  254,  226, ..., 1788, 1917, 1954],
            [ 319,  247,  244, ...,  990, 1184, 1331]],
    
           [[-999, -999, -999, ..., 2690, 2657, 2677],
            [-999, -999, -999, ..., 2620, 2638, 2707],
            [-999, -999, -999, ..., 2623, 2619, 2677],
            ...,
    ...
            ...,
            [ 374,  379,  360, ..., -999, -999, -999],
            [ 338,  344,  312, ..., -999, -999, -999],
            [ 295,  390,  369, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ...,  832,  759,  751],
            [-999, -999, -999, ...,  743,  715,  715],
            [-999, -999, -999, ...,  711,  694,  704],
            ...,
            [1905, 2202, 2051, ...,  794,  846,  840],
            [1496, 1940, 1932, ...,  782,  905,  791],
            [1182, 1697, 1765, ...,  414,  511,  662]],
    
           [[ 734,  763,  748, ...,  939,  815,  779],
            [ 726,  742,  739, ...,  821,  755,  728],
            [ 714,  729,  734, ...,  764,  711,  701],
            ...,
            [ 794,  796,  777, ..., -999, -999, -999],
            [ 766,  787,  768, ..., -999, -999, -999],
            [ 814,  862,  865, ..., -999, -999, -999]]], dtype=int16)

  • nbart_panchromatic
    (time, y, x)
    int16
    -999 -999 -999 ... -999 -999 -999

    units :

    1

    nodata :

    -999

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-999, -999, -999, ..., 2007, 2012, 2038],
            [-999, -999, -999, ..., 1968, 1988, 2034],
            [-999, -999, -999, ..., 1970, 1982, 2029],
            ...,
            [4886, 4887, 4879, ..., 3382, 3442, 3396],
            [4841, 4846, 4866, ..., 3439, 3529, 3472],
            [4823, 4857, 4853, ..., 3524, 3604, 3520]],
    
           [[1965, 1849, 1795, ..., 1170, 1085, 1095],
            [1595, 1604, 1611, ..., 1049, 1066, 1106],
            [1574, 1555, 1602, ..., 1066, 1068, 1120],
            ...,
            [ 553,  298,  273, ..., 1185, 1093,  995],
            [ 362,  253,  228, ...,  925,  919,  881],
            [ 321,  266,  273, ...,  583,  598,  621]],
    
           [[-999, -999, -999, ..., 1306, 1227, 1240],
            [-999, -999, -999, ..., 1189, 1204, 1239],
            [-999, -999, -999, ..., 1191, 1196, 1234],
            ...,
    ...
            ...,
            [ 232,  253,  221, ..., -999, -999, -999],
            [ 240,  230,  196, ..., -999, -999, -999],
            [ 223,  236,  222, ..., -999, -999, -999]],
    
           [[-999, -999, -999, ...,  441,  399,  398],
            [-999, -999, -999, ...,  393,  380,  389],
            [-999, -999, -999, ...,  377,  371,  386],
            ...,
            [1291, 1485, 1473, ...,  397,  448,  435],
            [ 997, 1366, 1378, ...,  413,  479,  497],
            [ 729, 1084, 1144, ...,  222,  273,  272]],
    
           [[ 374,  387,  399, ...,  370,  312,  309],
            [ 394,  413,  407, ...,  314,  302,  307],
            [ 397,  405,  413, ...,  300,  286,  311],
            ...,
            [ 735,  705,  673, ..., -999, -999, -999],
            [ 755,  747,  775, ..., -999, -999, -999],
            [ 734,  763,  773, ..., -999, -999, -999]]], dtype=int16)

  • oa_fmask
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    1

    nodata :

    0

    flags_definition :

    {'fmask': {'bits': [0, 1, 2, 3, 4, 5, 6, 7], 'values': {'0': 'nodata', '1': 'valid', '2': 'cloud', '3': 'shadow', '4': 'snow', '5': 'water'}, 'description': 'Fmask'}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            [0, 0, 0, ..., 2, 2, 2],
            ...,
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2],
            [2, 2, 2, ..., 2, 2, 2]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0],
            [3, 3, 3, ..., 0, 0, 0]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
    ...
            ...,
            [1, 1, 1, ..., 0, 0, 0],
            [1, 1, 1, ..., 0, 0, 0],
            [1, 1, 1, ..., 0, 0, 0]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [2, 2, 2, ..., 0, 0, 0],
            [2, 2, 2, ..., 0, 0, 0],
            [2, 2, 2, ..., 0, 0, 0]]], dtype=uint8)

  • oa_nbart_contiguity
    (time, y, x)
    uint8
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    1

    nodata :

    255

    flags_definition :

    {'contiguous': {'bits': [0], 'values': {'0': False, '1': True}}}

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
    ...
            ...,
            [1, 1, 1, ..., 0, 0, 0],
            [1, 1, 1, ..., 0, 0, 0],
            [1, 1, 1, ..., 0, 0, 0]],
    
           [[0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            [0, 0, 0, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 0, 0, 0],
            [1, 1, 1, ..., 0, 0, 0],
            [1, 1, 1, ..., 0, 0, 0]]], dtype=uint8)

  • oa_azimuthal_exiting
    (time, y, x)
    float32
    -80.12 -69.12 ... -74.58 -38.04

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -80.11553  ,  -69.122116 ,  -74.337006 , ...,   33.770046 ,
               97.201294 ,  133.90964  ],
            [ -98.09167  ,  -94.05396  ,  -87.45524  , ..., -135.61787  ,
              -76.76864  ,   76.9891   ],
            [-102.43699  , -115.73346  , -107.09284  , ..., -145.66154  ,
             -141.03618  , -102.94009  ],
            ...,
            [  18.39366  ,   13.645186 ,   17.93953  , ...,   60.979168 ,
               83.76587  ,   66.93567  ],
            [  19.606314 ,   11.669298 ,   11.353658 , ...,   88.5983   ,
               61.46648  ,   47.17998  ],
            [  11.797114 ,    2.8835816,    3.9532704, ...,   91.169556 ,
               22.845282 ,   38.14933  ]],
    
           [[  68.512245 ,   70.72293  ,   63.153164 , ...,  -57.384872 ,
              -57.144466 ,   -8.937615 ],
            [  57.491165 ,   53.669292 ,   51.882816 , ...,   62.990677 ,
               32.850204 , -101.98451  ],
            [  41.615105 ,   28.149704 ,   29.797287 , ...,   56.238945 ,
               50.34515  ,  -35.11987  ],
    ...
            [  17.934513 ,   13.017324 ,   16.885078 , ...,   59.4851   ,
               80.688286 ,   64.56885  ],
            [  18.617931 ,   10.572558 ,   10.177264 , ...,   84.03703  ,
               58.498463 ,   45.04802  ],
            [  10.611291 ,    1.6624858,    2.9239812, ...,   86.68579  ,
               21.029926 ,   36.18714  ]],
    
           [[  70.48015  ,   73.290184 ,   65.578514 , ...,  -57.312717 ,
              -56.889038 ,   -8.57699  ],
            [  59.72557  ,   56.965633 ,   55.223354 , ...,   62.880802 ,
               32.89343  , -102.08935  ],
            [  43.925644 ,   30.747004 ,   32.667336 , ...,   56.05834  ,
               50.242622 ,  -35.299603 ],
            ...,
            [ -57.612507 ,  -63.220844 ,  -60.561947 , ...,   41.13578  ,
               29.195576 ,   12.5532255],
            [ -58.60373  ,  -60.479248 ,  -56.80857  , ...,   34.061737 ,
              -24.82195  ,  -22.261066 ],
            [ -53.90224  ,  -52.506134 ,  -46.861496 , ...,  -24.654785 ,
              -74.57941  ,  -38.04392  ]]], dtype=float32)

  • oa_azimuthal_incident
    (time, y, x)
    float32
    -53.29 -43.81 ... 131.0 130.9

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ -53.29337  ,  -43.814816 ,  -49.677975 , ...,   73.61261  ,
               83.28998  , -150.34354  ],
            [ -70.001015 ,  -67.43494  ,  -62.690357 , ...,  -89.084045 ,
              -35.280712 ,  110.770065 ],
            [ -77.07651  ,  -90.91911  ,  -84.01256  , ...,  -97.06677  ,
             -102.89715  ,  -62.93791  ],
            ...,
            [  71.86952  ,   65.37238  ,   70.617676 , ...,  -37.93936  ,
             -127.73148  ,   55.0821   ],
            [  72.86513  ,   63.776108 ,   64.33563  , ...,  -86.956635 ,
               92.59873  ,  138.75403  ],
            [  65.48859  ,   55.617256 ,   58.376175 , ..., -145.5778   ,
               96.301636 ,  128.84294  ]],
    
           [[ -49.41655  ,  -40.01738  ,  -45.558876 , ...,   77.89437  ,
               87.38188  , -146.33037  ],
            [ -65.91871  ,  -64.03974  ,  -59.186226 , ...,  -84.63618  ,
              -31.004824 ,  115.21529  ],
            [ -72.751    ,  -87.00204  ,  -80.26874  , ...,  -92.55007  ,
              -98.49117  ,  -58.568077 ],
    ...
            [ 107.93615  ,  100.97654  ,  105.83655  , ...,   -8.137026 ,
              -94.735756 , -129.9145   ],
            [ 108.35325  ,   99.366486 ,  100.35032  , ...,  -55.37755  ,
              127.22215  ,  101.965576 ],
            [ 101.73554  ,   92.36351  ,   96.22875  , ..., -113.137856 ,
              131.77196  ,  130.79767  ]],
    
           [[ -22.005808 ,  -12.698394 ,  -18.891384 , ...,  105.34329  ,
              114.71673  , -119.122986 ],
            [ -38.658928 ,  -35.84669  ,  -31.474371 , ...,  -56.997562 ,
               -3.3867369,  142.56752  ],
            [ -46.36235  ,  -60.030754 ,  -53.270397 , ...,  -64.842125 ,
              -70.64825  ,  -30.67841  ],
            ...,
            [ 108.153114 ,  101.30291  ,  106.45137  , ...,   -8.112951 ,
              -96.180145 , -131.07863  ],
            [ 108.84681  ,   99.909035 ,  100.877205 , ...,  -56.142773 ,
              125.44492  ,  101.4136   ],
            [ 102.290886 ,   92.967926 ,   96.62845  , ..., -119.11456  ,
              130.99011  ,  130.91386  ]]], dtype=float32)

  • oa_combined_terrain_shadow
    (time, y, x)
    uint8
    1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1

    units :

    1

    nodata :

    255

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
    ...
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]],
    
           [[1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            ...,
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1],
            [1, 1, 1, ..., 1, 1, 1]]], dtype=uint8)

  • oa_exiting_angle
    (time, y, x)
    float32
    9.029 9.148 9.09 ... 8.228 10.61

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 9.028952 ,  9.148497 ,  9.090177 , ...,  3.2647538,
              3.62325  ,  4.391917 ],
            [ 8.894047 ,  8.901706 ,  8.9225235, ...,  3.2806778,
              3.5062037,  4.4214954],
            [ 8.756343 ,  8.535571 ,  8.580602 , ...,  3.3433805,
              3.560637 ,  4.636894 ],
            ...,
            [11.375915 , 11.375726 , 11.181578 , ...,  5.0770392,
              4.6290197,  6.9061594],
            [11.1641655, 11.834425 , 12.261691 , ...,  3.2524686,
              5.314759 ,  7.5270324],
            [12.537902 , 13.934173 , 14.685269 , ...,  3.791945 ,
              6.8892117,  7.5075154]],
    
           [[ 2.4255953,  2.4921818,  2.615967 , ...,  7.997434 ,
              7.613728 ,  6.8861303],
            [ 2.4387937,  2.5729144,  2.7521896, ...,  8.068149 ,
              7.777022 ,  6.803534 ],
            [ 2.8313277,  3.079027 ,  3.2421515, ...,  8.0514555,
              7.821932 ,  6.652463 ],
    ...
            [11.091274 , 11.119121 , 10.963267 , ...,  5.3241124,
              4.7204394,  6.954887 ],
            [10.960586 , 11.676337 , 12.139089 , ...,  3.3706117,
              5.221862 ,  7.456067 ],
            [12.401597 , 13.828    , 14.586696 , ...,  3.644003 ,
              6.704618 ,  7.39457  ]],
    
           [[ 2.6306045,  2.6835248,  2.800773 , ...,  8.209472 ,
              7.8253593,  7.0987763],
            [ 2.6306515,  2.7518692,  2.921237 , ...,  8.279742 ,
              7.988228 ,  7.016372 ],
            [ 3.00132  ,  3.2425504,  3.3996027, ...,  8.263643 ,
              8.033411 ,  6.864461 ],
            ...,
            [ 5.4920387,  5.0866117,  5.2272806, ..., 15.922826 ,
             13.239609 , 13.841619 ],
            [ 5.3568344,  5.4691668,  5.960857 , ..., 12.924013 ,
             10.294801 , 11.711468 ],
            [ 6.311668 ,  7.109315 ,  8.126275 , ...,  9.362463 ,
              8.227683 , 10.605143 ]]], dtype=float32)

  • oa_incident_angle
    (time, y, x)
    float32
    40.0 40.23 40.24 ... 60.33 58.04

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.998978, 40.23141 , 40.237175, ..., 37.26962 , 37.826603,
             38.7017  ],
            [39.77791 , 39.912277, 40.08276 , ..., 37.007404, 37.392212,
             38.432606],
            [39.88898 , 39.73803 , 39.905514, ..., 36.956375, 37.170578,
             38.289707],
            ...,
            [39.404842, 39.735092, 39.47829 , ..., 31.522882, 33.095287,
             32.230022],
            [39.361443, 39.853664, 39.843723, ..., 33.740055, 35.684074,
             34.42446 ],
            [39.81281 , 40.565845, 40.41699 , ..., 36.779434, 37.866028,
             35.635822]],
    
           [[40.41124 , 40.656933, 40.68124 , ..., 37.69816 , 38.263546,
             39.134884],
            [40.19523 , 40.337387, 40.529827, ..., 37.413853, 37.801704,
             38.827084],
            [40.35013 , 40.206455, 40.390034, ..., 37.354183, 37.558178,
             38.643627],
    ...
            [62.930397, 63.330765, 63.14946 , ..., 60.955334, 60.50269 ,
             58.317307],
            [62.99762 , 63.105022, 62.751194, ..., 61.70582 , 61.338573,
             59.052258],
            [62.523216, 62.42357 , 61.64701 , ..., 63.008038, 62.27948 ,
             59.97451 ]],
    
           [[65.03101 , 65.33157 , 65.40151 , ..., 62.691208, 63.24405 ,
             63.97428 ],
            [64.75233 , 64.99936 , 65.299866, ..., 62.302887, 62.67311 ,
             63.472343],
            [65.11423 , 65.08869 , 65.37001 , ..., 62.188828, 62.31943 ,
             63.061047],
            ...,
            [60.992832, 61.401543, 61.220016, ..., 59.172993, 58.6754  ,
             56.474434],
            [61.08193 , 61.194725, 60.85231 , ..., 59.91568 , 59.496246,
             57.21007 ],
            [60.61334 , 60.498722, 59.73377 , ..., 61.089355, 60.326176,
             58.04129 ]]], dtype=float32)

  • oa_relative_azimuth
    (time, y, x)
    float32
    32.23 32.23 32.23 ... 244.9 244.9

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[ 32.232582,  32.23285 ,  32.233112, ...,  32.693703,
              32.69397 ,  32.69425 ],
            [ 32.23295 ,  32.23321 ,  32.233475, ...,  32.69407 ,
              32.69434 ,  32.694626],
            [ 32.233315,  32.233578,  32.23384 , ...,  32.694443,
              32.694706,  32.694992],
            ...,
            [ 32.975964,  32.976234,  32.976505, ...,  33.447357,
              33.447643,  33.447918],
            [ 32.976337,  32.976612,  32.976875, ...,  33.447735,
              33.448017,  33.44829 ],
            [ 32.976727,  32.976997,  32.977264, ...,  33.44813 ,
              33.448418,  33.44869 ]],
    
           [[216.49684 , 216.49713 , 216.49739 , ..., 216.99608 ,
             216.99635 , 216.99666 ],
            [216.4972  , 216.49747 , 216.49776 , ..., 216.99643 ,
             216.99673 , 216.99702 ],
            [216.49756 , 216.49783 , 216.49811 , ..., 216.9968  ,
             216.99709 , 216.99738 ],
    ...
            [ 63.89906 ,  63.899326,  63.899593, ...,  64.36216 ,
              64.362434,  64.36271 ],
            [ 63.8992  ,  63.899467,  63.899734, ...,  64.3623  ,
              64.36257 ,  64.36284 ],
            [ 63.899353,  63.89962 ,  63.899887, ...,  64.36246 ,
              64.36273 ,  64.363   ]],
    
           [[244.12595 , 244.12622 , 244.1265  , ..., 244.60152 ,
             244.60179 , 244.60207 ],
            [244.12607 , 244.12636 , 244.12663 , ..., 244.60167 ,
             244.60194 , 244.60222 ],
            [244.12622 , 244.1265  , 244.12677 , ..., 244.6018  ,
             244.60208 , 244.60236 ],
            ...,
            [244.41234 , 244.41261 , 244.41287 , ..., 244.89056 ,
             244.89085 , 244.89113 ],
            [244.41248 , 244.41277 , 244.41302 , ..., 244.89072 ,
             244.89099 , 244.89128 ],
            [244.41264 , 244.41292 , 244.4132  , ..., 244.89087 ,
             244.89116 , 244.89145 ]]], dtype=float32)

  • oa_relative_slope
    (time, y, x)
    float32
    26.82 25.31 24.66 ... -154.4 -155.9

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[  26.822166,   25.307304,   24.659027, ...,   39.842567,
               32.689396,   26.666718],
            [  28.09065 ,   26.619028,   24.764885, ...,   46.53382 ,
               41.487926,   33.780968],
            [  25.360477,   24.81435 ,   23.08028 , ...,   48.594772,
               47.248375,   40.002182],
            ...,
            [  53.47586 ,   51.727196,   52.678143, ..., -169.05045 ,
               43.374695,  110.16484 ],
            [  53.258812,   52.106808,   52.98198 , ...,  116.92698 ,
               94.31888 ,  100.24516 ],
            [  53.69147 ,   52.733677,   54.422905, ...,   82.8719  ,
               73.45635 ,   90.69361 ]],
    
           [[-117.928795, -110.74031 , -108.71204 , ..., -144.39311 ,
             -141.10318 , -137.39276 ],
            [-123.409874, -117.70903 , -111.069046, ..., -147.62685 ,
             -145.64392 , -142.8002  ],
            [-114.366104, -115.15174 , -110.066025, ..., -148.78902 ,
             -148.83632 , -148.09137 ],
    ...
            [  90.00163 ,   87.95922 ,   88.95147 , ..., -137.75404 ,
               79.44801 ,   85.310936],
            [  89.73531 ,   88.79392 ,   90.17306 , ..., -171.83757 ,
              100.24897 ,  139.69766 ],
            [  91.12425 ,   90.70103 ,   93.30477 , ...,  119.795616,
              110.742035,  129.70384 ]],
    
           [[ -92.48596 ,  -85.98858 ,  -84.4699  , ..., -117.01634 ,
             -114.023766, -110.546005],
            [ -98.3845  ,  -92.812325,  -86.69772 , ..., -119.878365,
             -118.06905 , -115.34314 ],
            [ -90.287994,  -90.77776 ,  -85.93774 , ..., -120.90046 ,
             -120.89086 , -120.021965],
            ...,
            [ 165.76562 ,  164.52376 ,  167.01332 , ..., -119.38066 ,
             -131.91595 , -143.63185 ],
            [ 167.45055 ,  160.38828 ,  157.68578 , ..., -125.299614,
             -143.14157 , -153.54523 ],
            [ 156.19313 ,  145.47406 ,  143.48994 , ..., -134.84052 ,
             -154.4305  , -155.9489  ]]], dtype=float32)

  • oa_satellite_azimuth
    (time, y, x)
    float32
    100.1 100.1 100.1 ... 280.2 280.2

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[100.1332  , 100.13319 , 100.13318 , ..., 100.11447 ,
             100.114456, 100.11445 ],
            [100.13323 , 100.13322 , 100.13321 , ..., 100.1145  ,
             100.114494, 100.11449 ],
            [100.13326 , 100.13325 , 100.13324 , ..., 100.11453 ,
             100.114525, 100.11452 ],
            ...,
            [100.19855 , 100.19853 , 100.198524, ..., 100.179245,
             100.17924 , 100.17922 ],
            [100.19858 , 100.19857 , 100.198555, ..., 100.17928 ,
             100.179276, 100.17926 ],
            [100.198616, 100.1986  , 100.198586, ..., 100.17931 ,
             100.1793  , 100.17929 ]],
    
           [[280.09695 , 280.09692 , 280.09692 , ..., 280.07846 ,
             280.07843 , 280.07843 ],
            [280.097   , 280.09695 , 280.09695 , ..., 280.0785  ,
             280.0785  , 280.07846 ],
            [280.09702 , 280.09702 , 280.097   , ..., 280.07852 ,
             280.07852 , 280.0785  ],
    ...
            [100.230934, 100.23092 , 100.23091 , ..., 100.21154 ,
             100.21153 , 100.21152 ],
            [100.230965, 100.23096 , 100.23094 , ..., 100.21157 ,
             100.21156 , 100.21155 ],
            [100.231   , 100.23099 , 100.23097 , ..., 100.2116  ,
             100.211586, 100.21158 ]],
    
           [[280.12955 , 280.12955 , 280.12952 , ..., 280.11096 ,
             280.11096 , 280.11093 ],
            [280.12958 , 280.12958 , 280.12955 , ..., 280.111   ,
             280.111   , 280.111   ],
            [280.1296  , 280.1296  , 280.12958 , ..., 280.11102 ,
             280.11102 , 280.11102 ],
            ...,
            [280.19424 , 280.1942  , 280.19418 , ..., 280.17508 ,
             280.17508 , 280.17505 ],
            [280.19427 , 280.19424 , 280.1942  , ..., 280.1751  ,
             280.1751  , 280.1751  ],
            [280.1943  , 280.19427 , 280.19427 , ..., 280.17514 ,
             280.17514 , 280.17514 ]]], dtype=float32)

  • oa_satellite_view
    (time, y, x)
    float32
    9.022 9.02 9.017 ... 7.95 7.952

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[9.02193  , 9.019681 , 9.017433 , ..., 5.130571 , 5.1282773,
             5.125959 ],
            [9.021114 , 9.018867 , 9.016618 , ..., 5.1297407, 5.1274476,
             5.1251235],
            [9.0203   , 9.018051 , 9.015802 , ..., 5.128911 , 5.1266174,
             5.1242886],
            ...,
            [7.31853  , 7.3162613, 7.3139925, ..., 3.4020782, 3.3997495,
             3.3974464],
            [7.31763  , 7.3153787, 7.3131275, ..., 3.401222 , 3.3989155,
             3.3966124],
            [7.316651 , 7.314382 , 7.3121133, ..., 3.400168 , 3.397848 ,
             3.3955448]],
    
           [[2.1344194, 2.1367428, 2.1390662, ..., 6.0918517, 6.094148 ,
             6.096469 ],
            [2.1352227, 2.137546 , 2.1398695, ..., 6.092644 , 6.094941 ,
             6.097266 ],
            [2.1360261, 2.1383495, 2.140673 , ..., 6.0934367, 6.095733 ,
             6.0980635],
    ...
            [7.033692 , 7.0314198, 7.029148 , ..., 3.1135292, 3.1111996,
             3.108895 ],
            [7.0327897, 7.0305357, 7.0282817, ..., 3.1126723, 3.1103644,
             3.1080601],
            [7.0318093, 7.029537 , 7.0272655, ..., 3.111617 , 3.1092958,
             3.1069913]],
    
           [[2.3506095, 2.3529317, 2.355254 , ..., 6.304876 , 6.3071694,
             6.3094873],
            [2.3514132, 2.3537354, 2.3560576, ..., 6.305668 , 6.307962 ,
             6.3102846],
            [2.352217 , 2.3545392, 2.356861 , ..., 6.3064604, 6.3087544,
             6.311082 ],
            ...,
            [4.0200334, 4.0223446, 4.024656 , ..., 7.945772 , 7.948068 ,
             7.9503407],
            [4.020911 , 4.0232043, 4.0254974, ..., 7.946579 , 7.9488535,
             7.951126 ],
            [4.0218697, 4.0241804, 4.0264916, ..., 7.9475803, 7.9498687,
             7.9521413]]], dtype=float32)

  • oa_solar_azimuth
    (time, y, x)
    float32
    67.9 67.9 67.9 ... 35.28 35.28

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[67.90062 , 67.900345, 67.90006 , ..., 67.42077 , 67.42049 ,
             67.4202  ],
            [67.90028 , 67.90001 , 67.899734, ..., 67.42043 , 67.42015 ,
             67.41986 ],
            [67.89995 , 67.89967 , 67.8994  , ..., 67.4201  , 67.419815,
             67.41952 ],
            ...,
            [67.22258 , 67.2223  , 67.222015, ..., 66.73189 , 66.73159 ,
             66.73131 ],
            [67.222244, 67.221954, 67.22168 , ..., 66.73155 , 66.731255,
             66.73097 ],
            [67.221886, 67.2216  , 67.22132 , ..., 66.73118 , 66.73089 ,
             66.7306  ]],
    
           [[63.60011 , 63.59981 , 63.599514, ..., 63.082386, 63.082085,
             63.08177 ],
            [63.59979 , 63.59949 , 63.599194, ..., 63.082058, 63.081757,
             63.081444],
            [63.599464, 63.599167, 63.59887 , ..., 63.08173 , 63.08143 ,
             63.081112],
    ...
            [36.331875, 36.331596, 36.331314, ..., 35.84938 , 35.84909 ,
             35.84881 ],
            [36.331768, 36.33149 , 36.33121 , ..., 35.849274, 35.84899 ,
             35.84871 ],
            [36.331646, 36.331364, 36.331085, ..., 35.849144, 35.84886 ,
             35.84858 ]],
    
           [[36.0036  , 36.003315, 36.003025, ..., 35.50945 , 35.50916 ,
             35.508865],
            [36.003494, 36.00321 , 36.00292 , ..., 35.509342, 35.509052,
             35.50876 ],
            [36.003387, 36.0031  , 36.00281 , ..., 35.50923 , 35.50894 ,
             35.508648],
            ...,
            [35.7819  , 35.78161 , 35.78132 , ..., 35.284508, 35.284214,
             35.28392 ],
            [35.78178 , 35.781494, 35.781204, ..., 35.284397, 35.284107,
             35.283813],
            [35.781654, 35.781364, 35.781075, ..., 35.28426 , 35.28397 ,
             35.283676]]], dtype=float32)

  • oa_solar_zenith
    (time, y, x)
    float32
    39.49 39.49 39.49 ... 64.21 64.21

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[39.487972, 39.48771 , 39.48745 , ..., 39.04551 , 39.04525 ,
             39.044994],
            [39.48805 , 39.487785, 39.487526, ..., 39.045586, 39.045326,
             39.04507 ],
            [39.48812 , 39.48786 , 39.4876  , ..., 39.045662, 39.045403,
             39.045147],
            ...,
            [39.63633 , 39.63607 , 39.63581 , ..., 39.195976, 39.195724,
             39.19546 ],
            [39.636402, 39.636143, 39.635883, ..., 39.196053, 39.1958  ,
             39.19554 ],
            [39.636463, 39.636204, 39.635944, ..., 39.196114, 39.195854,
             39.195595]],
    
           [[39.84592 , 39.845665, 39.845406, ..., 39.414013, 39.413757,
             39.413506],
            [39.846012, 39.845757, 39.8455  , ..., 39.41411 , 39.413853,
             39.413605],
            [39.846107, 39.84585 , 39.845596, ..., 39.414204, 39.413948,
             39.4137  ],
    ...
            [66.40985 , 66.40967 , 66.40948 , ..., 66.09807 , 66.09789 ,
             66.0977  ],
            [66.41005 , 66.40986 , 66.409676, ..., 66.09827 , 66.09808 ,
             66.0979  ],
            [66.41024 , 66.41005 , 66.40987 , ..., 66.09846 , 66.098274,
             66.09808 ]],
    
           [[64.118195, 64.118004, 64.11782 , ..., 63.808098, 63.807907,
             63.80773 ],
            [64.11839 , 64.11821 , 64.11803 , ..., 63.808296, 63.80811 ,
             63.807938],
            [64.1186  , 64.11841 , 64.118225, ..., 63.8085  , 63.80831 ,
             63.80814 ],
            ...,
            [64.51843 , 64.51825 , 64.51807 , ..., 64.209854, 64.20968 ,
             64.209496],
            [64.51863 , 64.51845 , 64.518265, ..., 64.21006 , 64.209885,
             64.2097  ],
            [64.51882 , 64.51864 , 64.518456, ..., 64.21025 , 64.21007 ,
             64.209885]]], dtype=float32)

  • oa_time_delta
    (time, y, x)
    float32
    -0.4956 -0.4971 ... 1.01 1.008

    units :

    1

    nodata :

    nan

    crs :

    EPSG:3577

    grid_mapping :

    spatial_ref

    array([[[-0.49559563, -0.4970854 , -0.49857512, ..., -2.9306152 ,
             -2.9321241 , -2.9334474 ],
            [-0.49143195, -0.49292168, -0.49441144, ..., -2.9264584 ,
             -2.9279675 , -2.929252  ],
            [-0.48726824, -0.48875797, -0.49024773, ..., -2.9223018 ,
             -2.923811  , -2.9250565 ],
            ...,
            [ 7.86643   ,  7.864933  ,  7.863436  , ...,  5.4181333 ,
              5.416802  ,  5.4152856 ],
            [ 7.870569  ,  7.869079  ,  7.867589  , ...,  5.422319  ,
              5.420964  ,  5.4194474 ],
            [ 7.874677  ,  7.8731804 ,  7.8716836 , ...,  5.4264264 ,
              5.4250336 ,  5.423517  ]],
    
           [[-4.8207088 , -4.8221984 , -4.823688  , ..., -7.255672  ,
             -7.2571807 , -7.2585034 ],
            [-4.816544  , -4.8180337 , -4.8195233 , ..., -7.251514  ,
             -7.2530227 , -7.254307  ],
            [-4.812379  , -4.8138685 , -4.815358  , ..., -7.247356  ,
             -7.248864  , -7.2501097 ],
    ...
            [ 7.7582965 ,  7.7567964 ,  7.755297  , ...,  5.3051767 ,
              5.3038425 ,  5.3023233 ],
            [ 7.762434  ,  7.7609415 ,  7.759449  , ...,  5.309362  ,
              5.3080034 ,  5.306484  ],
            [ 7.766541  ,  7.765042  ,  7.7635417 , ...,  5.313467  ,
              5.3120713 ,  5.3105526 ]],
    
           [[-4.904923  , -4.906416  , -4.9079084 , ..., -7.3447366 ,
             -7.346248  , -7.347574  ],
            [-4.900759  , -4.9022517 , -4.9037447 , ..., -7.3405795 ,
             -7.342091  , -7.343378  ],
            [-4.896595  , -4.898088  , -4.8995805 , ..., -7.3364224 ,
             -7.3379345 , -7.3391824 ],
            ...,
            [ 3.4559293 ,  3.4544291 ,  3.4529293 , ...,  1.0027968 ,
              1.0014625 ,  0.99994326],
            [ 3.4600687 ,  3.4585757 ,  3.4570827 , ...,  1.0069828 ,
              1.0056249 ,  1.0041058 ],
            [ 3.4641771 ,  3.4626772 ,  3.4611773 , ...,  1.01109   ,
              1.0096943 ,  1.0081753 ]]], dtype=float32)

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2020-02-08 23:56:55.714845', '2020-02-16 00:03:05.179449',
                   '2020-02-24 23:56:51.968040', '2020-03-03 00:03:00.297499',
                   '2020-03-11 23:56:45.657477', '2020-03-19 00:02:52.920718',
                   '2020-03-27 23:56:36.905801', '2020-04-04 00:02:43.534168',
                   '2020-04-12 23:56:29.492153', '2020-04-20 00:02:37.218009',
                   '2020-04-28 23:56:21.738632', '2020-05-06 00:02:28.332833',
                   '2020-05-14 23:56:17.682743', '2020-05-22 00:02:29.991429',
                   '2020-05-30 23:56:21.787385', '2020-06-07 00:02:37.301210',
                   '2020-06-15 23:56:31.825059', '2020-06-23 00:02:46.402339',
                   '2020-07-01 23:56:39.787669', '2020-07-09 00:02:53.484642',
                   '2020-07-17 23:56:45.798382', '2020-07-25 00:02:58.653767'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • y
    PandasIndex

    PandasIndex(Float64Index([-3988395.0, -3988425.0, -3988455.0, -3988485.0, -3988515.0,
                  -3988545.0, -3988575.0, -3988605.0, -3988635.0, -3988665.0,
                  ...
                  -4048485.0, -4048515.0, -4048545.0, -4048575.0, -4048605.0,
                  -4048635.0, -4048665.0, -4048695.0, -4048725.0, -4048755.0],
                 dtype='float64', name='y', length=2013))

  • x
    PandasIndex

    PandasIndex(Float64Index([1327605.0, 1327635.0, 1327665.0, 1327695.0, 1327725.0, 1327755.0,
                  1327785.0, 1327815.0, 1327845.0, 1327875.0,
                  ...
                  1378575.0, 1378605.0, 1378635.0, 1378665.0, 1378695.0, 1378725.0,
                  1378755.0, 1378785.0, 1378815.0, 1378845.0],
                 dtype='float64', name='x', length=1709))

• Attributes: (2)

  crs :

  EPSG:3577

  grid_mapping :

  spatial_ref

How fast this step is will depend on how many cores are in your Jupyter notebook's local cluster. In real world scenarios, an 8-core cluster the datacube.load() this may take between 1/4 or 1/6 of the time compared to without dask (not shown) depending on many factors. This is great!

Why not 1/8 of the time?

Dask has overheads, and datacube.load() itself is IO limited. There are all sorts of things that result in limits and part of the art of parallel computing is tuning your algorithm to reduce the impact of these and achieve greater performance. As we scale up this example we'll explore some of these.

Tip: recent updates to Dask have greatly improved performance and we are now seeing more substantial performance gains, more in line with the increase in cores.

Do not always expect 8x as many cores to produce 8x the speed up. Algorithms can be tuned to perform better (or worse) as scale increases. This is part of the art of parallel programming. Dask does it's best, and you can often do better.

Exploiting delayed tasks

Now let's repeat the full example, with NDVI calculation and masking, in a single cell with dask and compute to load the data in. We get the total time for later comparison.

Most of the time (not shown) is in the data load and the NDVI calculation is < 1 second.

To ensure comparable timings, we will .restart() the Dask cluster. This makes sure that we aren't just seeing performance gains for data caching.

Note that this may show some Restarting worker warnings. That is ok and it is just telling you that each of the four workers in the cluster are restarting.

if client is None:
    cluster = LocalCluster()
    client = Client(cluster)
    configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);
    display(notebook_utils.localcluster_dashboard(client=client, server=easi.hub))
else:
    client.restart()

'https://hub.csiro.easi-eo.solutions/user/csiro-csiro-aad_pag064@csiro.au/proxy/8787/status'

%%time

chunks = {"time":1}

dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, 
            group_by=group_by,
        )
actual_dataset = dataset.compute() ### Compute the dataset ###

# Identify pixels that don't have cloud, cloud shadow or water
cloud_free_mask = masking.make_mask(actual_dataset[qa_band], **qa_mask)

# Apply the mask
cloud_free = actual_dataset.where(cloud_free_mask)

# Calculate the components that make up the NDVI calculation
band_diff = cloud_free[alias['nir']] - cloud_free[alias['red']]
band_sum = cloud_free[alias['nir']] + cloud_free[alias['red']]
# Calculate NDVI and store it as a measurement in the original dataset ta da
ndvi = None
ndvi = band_diff / band_sum

CPU times: user 5.69 s, sys: 2.67 s, total: 8.36 s
Wall time: 31.8 s

Quicker but can do better...

Data and computational locality

When compute() is called dask not only executes all the tasks but it consolidates all the distributed chunks back into a normal array on the client machine - in this case the notebook's kernel. In the previous cell we have two variables that both refer to the data we are loading:

1. dataset refers to the delayed version of the data. The delayed tasks and the chunks that make it up will be on the cluster
2. actual_dataset refers to the actual array in the notebook kernel memory after execution of the tasks. The actual_dataset is a complete array in memory in the notebook kernel (__on the _client___).

So in the previous cell everything after the actual_dataset = dataset.compute() line is computed in the Jupyter kernel and doesn't use the dask cluster at all for computation.

If we shift the location of this compute() call we can perform more tasks in parallel on the dask cluster.

Tip: Locality is an important concept and applies to both data and computation

Now let's repeat the load and NDVI calculation but this time rather than compute() on the full dataset we'll run the compute at the cloud masking step (cloud_free = dataset.where(cloud__free_mask).compute()) so the masking operation can be performed in parallel. Let's see what the impact is...

if client is None:
    cluster = LocalCluster()
    client = Client(cluster)
    configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);
    display(notebook_utils.localcluster_dashboard(client=client, server=easi.hub))
else:
    client.restart()

'https://hub.csiro.easi-eo.solutions/user/csiro-csiro-aad_pag064@csiro.au/proxy/8787/status'

%%time

chunks = {"time":1}

dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, 
            group_by=group_by,
        )

# Identify pixels that are either "valid", "water" or "snow"
cloud_free_mask = masking.make_mask(dataset[qa_band], **qa_mask)

# Apply the mask
cloud_free = dataset.where(cloud_free_mask).compute()    ### COMPUTE MOVED HERE ###

# Calculate the components that make up the NDVI calculation
band_diff = cloud_free[alias['nir']] - cloud_free[alias['red']]
band_sum = cloud_free[alias['nir']] + cloud_free[alias['red']]
# Calculate NDVI and store it as a measurement in the original dataset ta da
ndvi = None
ndvi = band_diff / band_sum
actual_ndvi = ndvi

CPU times: user 3.3 s, sys: 2.87 s, total: 6.16 s
Wall time: 30.3 s

A few seconds quicker but not that different. This isn't too surprising since the masking operation is pretty quick (it's all numpy) and the data load is the bulk of the processing.

Dask can see the entire task graph for both load and mask computation. As a result some of the computation can be performed concurrently with file IO, and CPUs are busier as a result, so it will be slightly faster in practice but with IO dominating we won't see much overall improvement.

With compute on the algorithm

Perhaps doing more of the calculation on the cluster will help. Let's also move ndvi.compute() so the entire calculation is done on the cluster and only the final result returned to the client.

if client is None:
    cluster = LocalCluster()
    client = Client(cluster)
    configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);
    display(notebook_utils.localcluster_dashboard(client=client, server=easi.hub))
else:
    client.restart()

'https://hub.csiro.easi-eo.solutions/user/csiro-csiro-aad_pag064@csiro.au/proxy/8787/status'

%%time

chunks = {"time":1}

dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, 
            group_by=group_by,
        )

# Identify pixels that don't have cloud, cloud shadow or water
cloud_free_mask = masking.make_mask(dataset[qa_band], **qa_mask)

# Apply the mask
cloud_free = dataset.where(cloud_free_mask)

# Calculate the components that make up the NDVI calculation
band_diff = cloud_free[alias['nir']] - cloud_free[alias['red']]
band_sum = cloud_free[alias['nir']] + cloud_free[alias['red']]
# Calculate NDVI and store it as a measurement in the original dataset ta da
ndvi = None
ndvi = band_diff / band_sum
actual_ndvi = ndvi.compute()    ### COMPUTE MOVED HERE ###

CPU times: user 755 ms, sys: 257 ms, total: 1.01 s
Wall time: 6.03 s

Now we are seeing a huge difference!

You may be thinking "Hold on a sec, the NDVI calculation is pretty quick in this example with such a small dataset, why such a big difference?" - and you'd be right. There is more going on.

Remember that dataset is a task graph with delayed tasks waiting to be executed when the result is required. In the example dataset, there are many data variables available but only 3 are used to produce the ndvi (qa_band, red and nir). As a result dask doesn't load the other variables and because computation time in this case is mostly IO related the execution time is a lot faster.

With selected measurements

Of course we can save dask the trouble of figuring this out on our behalf and only load() the measurements we need in the first place. Let's check that now, we should see a similar performance figure.

if client is None:
    cluster = LocalCluster()
    client = Client(cluster)
    configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);
    display(notebook_utils.localcluster_dashboard(client=client, server=easi.hub))
else:
    client.restart()

2023-06-06 12:41:28,483 - distributed.nanny - WARNING - Restarting worker
2023-06-06 12:41:28,498 - distributed.nanny - WARNING - Restarting worker
2023-06-06 12:41:28,499 - distributed.nanny - WARNING - Restarting worker
2023-06-06 12:41:28,507 - distributed.nanny - WARNING - Restarting worker

%%time

chunks = {"time":1}
measurements = [alias[x] for x in ['qa_band', 'red', 'nir']]

dataset = None # clear results from any previous runs
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, 
            group_by=group_by,
        )

# Identify pixels that don't have cloud, cloud shadow or water
cloud_free_mask = masking.make_mask(dataset[qa_band], **qa_mask)
# Apply the mask
cloud_free = dataset.where(cloud_free_mask)

# Calculate the components that make up the NDVI calculation
band_diff = cloud_free[alias['nir']] - cloud_free[alias['red']]
band_sum = cloud_free[alias['nir']] + cloud_free[alias['red']]
# Calculate NDVI and store it as a measurement in the original dataset ta da
ndvi = None
ndvi = band_diff / band_sum
actual_ndvi = ndvi.compute()

CPU times: user 387 ms, sys: 192 ms, total: 579 ms
Wall time: 4.92 s

Pretty similar as expected, but again, a slight improvement because now there are less overheads and a smaller task graph. Now it can pay to give dask a hand and not have the task graph cluttered with tasks you are not going to use. Still it's nice to see that dask can save you some time by only computing what is required when you need it.

A quick check on the task graph

For completeness we will take a look at the task graph for the full calculation, all the way to the NDVI result. Given the complexity of the full graph we'll simplify it to 2 time observations like we did when the task graph was introduced previously.

set_time = (set_time[0], parse(set_time[0]) + relativedelta(weeks=3))
# set_time = ("2021-01-01", "2021-01-14")
set_time

('2020-02-01', datetime.datetime(2020, 2, 22, 0, 0))

if client is None:
    cluster = LocalCluster()
    client = Client(cluster)
    configure_s3_access(aws_unsigned=False, requester_pays=True, client=client);
    display(notebook_utils.localcluster_dashboard(client=client, server=easi.hub))
else:
    client.restart()

2023-06-06 12:42:20,790 - distributed.nanny - WARNING - Restarting worker
2023-06-06 12:42:20,796 - distributed.nanny - WARNING - Restarting worker
2023-06-06 12:42:20,847 - distributed.nanny - WARNING - Restarting worker
2023-06-06 12:42:20,853 - distributed.nanny - WARNING - Restarting worker

%%time
dataset = None # clear results from any previous runs
measurements = [alias[x] for x in ['qa_band', 'red', 'nir']]
dataset = dc.load(
            product=product,
            x=study_area_lon,
            y=study_area_lat,
            time=set_time,
            measurements=measurements,
            resampling=resampling,
            output_crs=set_crs,
            resolution=set_resolution,
            dask_chunks = chunks, 
            group_by=group_by,
        )

# Identify pixels that don't have cloud, cloud shadow or water
cloud_free_mask = masking.make_mask(dataset[qa_band], **qa_mask)
# Apply the mask
cloud_free = dataset.where(cloud_free_mask)

# Calculate the components that make up the NDVI calculation
band_diff = cloud_free[alias['nir']] - cloud_free[alias['red']]
band_sum = cloud_free[alias['nir']] + cloud_free[alias['red']]
# Calculate NDVI and store it as a measurement in the original dataset ta da
ndvi = None
ndvi = band_diff / band_sum

CPU times: user 48.5 ms, sys: 3.19 ms, total: 51.7 ms
Wall time: 72 ms

ndvi.data.visualize()

The computation flows from bottom to top in the task graph. You can see there are two main paths, one for each time (since the time chunk is length 1). You can also see the three data sources are loaded independently. After that it gets a little more difficult to follow but you can see qa_band being used to produce the mask (and_, eq). Then combined via the where function with other two datasets. Then finally the NDVI calculation - a sub, add and divide (truediv).

Dask has lots of internal optimizations that it uses to help identify the dependencies and parallel components of a task graph. Sometimes it will reorder or prune operations where possible to further optimise (for example, not loading data variables that aren't used in the NDVI calculation).

Tip: The task graph can be complex but it is a useful tool in understanding your algorithm and how it scales.

Be a good dask user - Clean up the cluster resources

client.close()

cluster.close()